2023-11-28T10:56:47.421 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T10:56:47.422 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} -- /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/orig.config.yaml 2023-11-28T10:56:47.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T10:56:47.563 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469958' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9437 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_10:48:03 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T10:56:47.563 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa; will attempt to use it 2023-11-28T10:56:47.564 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks 2023-11-28T10:56:47.564 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T10:56:47.565 INFO:teuthology.task.internal:Checking packages... 2023-11-28T10:56:47.591 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T10:56:47.591 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T10:56:47.592 INFO:teuthology.packaging:ref: None 2023-11-28T10:56:47.592 INFO:teuthology.packaging:tag: None 2023-11-28T10:56:47.592 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T10:56:47.592 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T10:56:47.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T10:56:47.733 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T10:56:47.734 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T10:56:47.740 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T10:56:47.740 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T10:56:47.746 INFO:teuthology.task.internal:Saving configuration 2023-11-28T10:56:47.764 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T10:56:47.771 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T10:56:47.796 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:57.408838', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:56:47.822 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:57.410360', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:56:47.822 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T10:56:47.829 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T10:56:47.829 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T10:56:47.829 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T10:56:47.900 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5ffafd0a60>, signals=[15]) 2023-11-28T10:56:47.900 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T10:56:47.906 INFO:teuthology.task.internal:Opening connections... 2023-11-28T10:56:47.906 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T10:56:47.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:56:47.977 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T10:56:47.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:56:48.046 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T10:56:48.052 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-11-28T10:56:48.074 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-11-28T10:56:48.074 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-11-28T10:56:48.135 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:56:48.135 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:56:48.135 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2023-11-28T10:56:48.135 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2023-11-28T10:56:48.135 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:56:48.136 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:56:48.137 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:56:48.137 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-11-28T10:56:48.162 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-11-28T10:56:48.183 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-11-28T10:56:48.183 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-11-28T10:56:48.242 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:56:48.242 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:56:48.242 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-11-28T10:56:48.242 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:56:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T10:56:48.244 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:56:48.244 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:56:48.244 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:56:48.244 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:56:48.244 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-11-28T10:56:48.262 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T10:56:48.273 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T10:56:48.279 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T10:56:48.279 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:56:48.282 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:56:48.302 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T10:56:48.308 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T10:56:48.308 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:56:48.343 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:56:48.375 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T10:56:48.429 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T10:56:48.445 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T10:56:48.445 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-28T10:56:48.445 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-28T10:56:48.446 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:56:48.446 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:56:48.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:56:48.466 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T10:56:48.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:56:48.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T10:56:48.650 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:56:48.650 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T10:56:48.650 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-11-28T10:56:48.670 INFO:teuthology.orchestra.run.smithi157.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:56:48.670 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:56:48.670 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T10:56:48.671 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum upgrade -y linux-firmware 2023-11-28T10:56:48.701 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:56:48.702 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T10:56:48.702 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2023-11-28T10:56:48.722 INFO:teuthology.orchestra.run.smithi053.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:56:48.722 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:56:48.722 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T10:56:48.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum upgrade -y linux-firmware 2023-11-28T10:56:49.867 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-11-28T10:56:49.868 INFO:teuthology.orchestra.run.smithi157.stdout:Unable to read consumer identity 2023-11-28T10:56:49.941 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-28T10:56:49.942 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2023-11-28T10:56:51.335 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-28T10:56:51.523 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-11-28T10:56:52.060 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-28T10:56:55.635 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 618 kB/s | 24 kB 00:00 2023-11-28T10:56:56.223 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 16 MB 00:04 2023-11-28T10:56:56.376 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-11-28T10:56:56.377 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-11-28T10:56:56.377 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-11-28T10:56:56.507 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T10:56:56.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:56:56.651 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T10:56:56.651 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T10:56:56.654 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2023-11-28T10:56:59.898 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 405 kB/s | 24 kB 00:00 2023-11-28T10:57:00.677 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-28T10:57:00.678 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2023-11-28T10:57:00.678 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-28T10:57:00.782 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T10:57:00.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:57:00.927 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T10:57:00.928 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T10:57:00.930 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi053.front.sepia.ceph.com to complete... 2023-11-28T10:57:31.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28 10:57:31 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T10:57:31.136 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2023-11-28 10:57:31-- 2023-11-28T10:57:31.137 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 34s 2023-11-28T10:57:31.137 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 365M in 34s (10.7 MB/s) 2023-11-28T10:57:31.138 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T10:57:31.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T10:57:32.738 INFO:teuthology.orchestra.run.smithi157.stdout:Verifying... ######################################## 2023-11-28T10:57:33.281 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing... ######################################## 2023-11-28T10:57:33.645 INFO:teuthology.orchestra.run.smithi157.stdout:Updating / installing... 2023-11-28T10:57:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28 10:57:33 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T10:57:33.929 INFO:teuthology.orchestra.run.smithi053.stderr:FINISHED --2023-11-28 10:57:33-- 2023-11-28T10:57:33.929 INFO:teuthology.orchestra.run.smithi053.stderr:Total wall clock time: 33s 2023-11-28T10:57:33.929 INFO:teuthology.orchestra.run.smithi053.stderr:Downloaded: 1 files, 365M in 33s (11.2 MB/s) 2023-11-28T10:57:33.970 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T10:57:33.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T10:57:36.113 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying... ######################################## 2023-11-28T10:57:36.180 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing... ######################################## 2023-11-28T10:57:36.272 INFO:teuthology.orchestra.run.smithi053.stdout:Updating / installing... 2023-11-28T10:58:11.854 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T10:58:15.989 INFO:teuthology.orchestra.run.smithi157.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T10:58:17.162 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T10:58:18.821 INFO:teuthology.orchestra.run.smithi053.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T10:58:35.631 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T10:58:35.631 DEBUG:teuthology.orchestra.run.smithi157:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T10:58:35.699 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T10:58:35.700 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T10:58:35.701 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T10:58:35.702 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T10:58:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T10:58:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T10:58:35.705 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T10:58:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T10:58:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T10:58:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T10:58:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T10:58:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T10:58:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T10:58:35.712 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T10:58:35.713 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T10:58:35.714 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T10:58:35.715 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T10:58:35.716 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T10:58:35.717 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T10:58:35.718 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T10:58:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T10:58:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T10:58:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T10:58:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T10:58:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T10:58:35.724 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T10:58:35.725 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T10:58:35.726 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T10:58:35.727 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T10:58:35.728 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T10:58:35.729 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T10:58:35.730 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T10:58:35.731 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T10:58:35.732 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T10:58:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T10:58:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T10:58:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T10:58:35.736 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T10:58:35.737 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T10:58:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T10:58:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T10:58:35.740 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T10:58:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T10:58:35.742 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T10:58:35.743 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T10:58:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T10:58:35.745 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T10:58:35.746 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T10:58:35.747 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T10:58:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T10:58:35.749 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T10:58:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T10:58:35.751 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T10:58:35.751 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T10:58:35.751 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T10:58:35.751 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T10:58:35.751 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T10:58:35.752 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T10:58:35.753 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T10:58:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T10:58:35.755 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T10:58:35.756 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T10:58:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T10:58:35.758 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T10:58:35.759 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T10:58:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T10:58:35.761 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T10:58:35.762 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T10:58:35.763 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T10:58:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T10:58:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T10:58:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T10:58:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T10:58:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T10:58:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T10:58:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T10:58:35.771 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T10:58:35.772 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T10:58:35.773 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T10:58:35.774 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T10:58:35.775 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T10:58:35.776 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T10:58:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T10:58:35.778 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T10:58:35.779 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T10:58:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T10:58:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T10:58:35.782 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T10:58:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T10:58:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T10:58:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T10:58:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T10:58:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T10:58:35.784 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T10:58:35.785 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T10:58:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T10:58:35.787 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T10:58:35.788 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T10:58:35.789 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T10:58:35.790 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T10:58:35.791 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T10:58:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T10:58:35.793 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T10:58:35.794 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T10:58:35.795 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T10:58:35.796 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T10:58:35.797 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T10:58:35.798 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T10:58:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T10:58:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T10:58:35.801 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T10:58:35.802 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T10:58:35.803 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T10:58:35.804 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T10:58:35.805 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T10:58:35.806 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T10:58:35.807 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T10:58:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T10:58:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T10:58:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T10:58:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T10:58:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T10:58:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T10:58:35.814 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T10:58:35.815 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T10:58:35.816 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T10:58:35.817 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T10:58:35.818 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T10:58:35.819 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T10:58:35.820 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T10:58:35.821 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T10:58:35.822 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T10:58:35.823 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T10:58:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T10:58:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T10:58:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T10:58:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T10:58:35.828 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T10:58:35.829 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T10:58:35.830 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T10:58:35.831 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T10:58:35.832 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T10:58:35.833 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T10:58:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T10:58:35.835 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T10:58:35.836 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T10:58:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T10:58:35.838 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T10:58:35.839 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T10:58:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T10:58:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T10:58:35.842 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T10:58:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T10:58:35.844 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T10:58:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T10:58:35.846 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T10:58:35.847 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T10:58:35.848 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T10:58:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T10:58:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T10:58:35.851 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T10:58:35.852 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T10:58:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T10:58:35.854 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T10:58:35.855 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T10:58:35.856 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T10:58:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T10:58:35.858 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T10:58:35.859 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T10:58:35.860 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T10:58:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T10:58:35.862 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T10:58:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T10:58:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T10:58:35.865 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T10:58:35.866 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T10:58:35.867 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T10:58:35.868 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T10:58:35.869 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T10:58:35.870 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T10:58:35.874 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T10:58:35.875 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T10:58:35.875 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T10:58:35.875 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T10:58:35.875 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T10:58:35.875 DEBUG:teuthology.orchestra.run.smithi157:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:35.902 INFO:teuthology.orchestra.run.smithi157.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T10:58:35.903 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:35.904 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-11-28T10:58:35.904 INFO:teuthology.orchestra.run.smithi157.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:35.904 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T10:58:35.904 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:35.905 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-11-28T10:58:35.905 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.02 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 123.el8 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Group : System Environment/Base 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 9265450 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-28T10:58:35.980 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Relocations : (not relocatable) 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : Red Hat, Inc. 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : Red Hat, Inc. 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T10:58:35.981 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2023-11-28T10:58:35.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T10:58:35.982 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T10:58:35.982 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T10:58:35.983 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 6.6.0-g8247bbb5b470 2023-11-28T10:58:35.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T10:58:40.966 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2023-11-28T10:58:42.725 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T10:58:42.725 DEBUG:teuthology.orchestra.run.smithi053:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T10:58:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T10:58:42.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T10:58:42.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T10:58:42.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T10:58:42.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T10:58:42.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T10:58:42.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T10:58:42.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T10:58:42.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T10:58:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T10:58:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T10:58:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T10:58:42.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T10:58:42.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T10:58:42.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T10:58:42.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T10:58:42.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T10:58:42.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T10:58:42.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T10:58:42.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T10:58:42.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T10:58:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T10:58:42.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T10:58:42.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T10:58:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T10:58:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T10:58:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T10:58:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T10:58:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T10:58:42.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T10:58:42.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T10:58:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T10:58:42.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T10:58:42.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T10:58:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T10:58:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T10:58:42.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T10:58:42.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T10:58:42.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T10:58:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T10:58:42.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T10:58:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T10:58:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T10:58:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T10:58:42.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T10:58:42.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T10:58:42.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T10:58:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T10:58:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T10:58:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T10:58:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T10:58:42.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T10:58:42.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T10:58:42.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T10:58:42.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T10:58:42.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T10:58:42.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T10:58:42.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T10:58:42.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T10:58:42.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T10:58:42.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T10:58:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T10:58:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T10:58:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T10:58:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T10:58:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T10:58:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T10:58:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T10:58:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T10:58:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T10:58:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T10:58:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T10:58:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T10:58:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T10:58:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T10:58:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T10:58:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T10:58:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T10:58:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T10:58:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T10:58:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T10:58:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T10:58:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T10:58:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T10:58:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T10:58:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T10:58:42.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T10:58:42.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T10:58:42.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T10:58:42.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T10:58:42.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T10:58:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T10:58:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T10:58:42.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T10:58:42.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T10:58:42.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T10:58:42.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T10:58:42.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T10:58:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T10:58:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T10:58:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T10:58:42.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T10:58:42.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T10:58:42.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T10:58:42.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T10:58:42.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T10:58:42.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T10:58:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T10:58:42.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T10:58:42.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T10:58:42.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T10:58:42.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T10:58:42.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T10:58:42.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T10:58:42.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T10:58:42.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T10:58:42.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T10:58:42.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T10:58:42.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T10:58:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T10:58:42.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T10:58:42.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T10:58:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T10:58:42.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T10:58:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T10:58:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T10:58:42.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T10:58:42.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T10:58:42.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T10:58:42.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T10:58:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T10:58:42.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T10:58:42.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T10:58:42.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T10:58:42.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T10:58:42.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T10:58:42.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T10:58:42.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T10:58:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T10:58:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T10:58:42.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T10:58:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T10:58:42.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T10:58:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T10:58:42.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T10:58:42.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T10:58:42.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T10:58:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T10:58:42.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T10:58:42.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T10:58:42.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T10:58:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T10:58:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T10:58:42.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T10:58:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T10:58:42.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T10:58:42.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T10:58:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T10:58:42.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T10:58:42.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T10:58:42.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T10:58:42.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T10:58:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T10:58:42.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T10:58:42.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T10:58:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T10:58:42.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T10:58:42.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T10:58:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T10:58:42.955 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T10:58:42.955 DEBUG:teuthology.orchestra.run.smithi053:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:42.983 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:42.984 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-11-28T10:58:42.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.02 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 123.el8 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Group : System Environment/Base 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 9265450 2023-11-28T10:58:43.067 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Relocations : (not relocatable) 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : Red Hat, Inc. 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : Red Hat, Inc. 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T10:58:43.068 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T10:58:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T10:58:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T10:58:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2023-11-28T10:58:43.069 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T10:58:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T10:58:43.070 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T10:58:43.070 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 6.6.0-g8247bbb5b470 2023-11-28T10:58:43.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T10:58:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:done 2023-11-28T10:58:43.462 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-11-28T10:58:43.484 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.wtqZCiRSgV 2023-11-28T10:58:43.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wtqZCiRSgV 2023-11-28T10:58:43.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.wtqZCiRSgV 2023-11-28T10:58:43.708 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.wtqZCiRSgV is 6KB 2023-11-28T10:58:43.756 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.wtqZCiRSgV 2023-11-28T10:58:43.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T10:58:43.855 INFO:teuthology.orchestra.run.smithi157.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-28T10:58:43.856 INFO:teuthology.orchestra.run.smithi157.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-28T10:58:43.856 INFO:teuthology.orchestra.run.smithi157.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-28T10:58:43.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-28T10:58:44.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2023-11-28T10:58:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2023-11-28T10:58:48.205 INFO:teuthology.orchestra.run.smithi053.stderr:done 2023-11-28T10:58:48.207 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2023-11-28T10:58:48.225 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.T6iHvIiNnb 2023-11-28T10:58:48.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.T6iHvIiNnb 2023-11-28T10:58:48.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.T6iHvIiNnb 2023-11-28T10:58:48.451 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.T6iHvIiNnb is 6KB 2023-11-28T10:58:48.499 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.T6iHvIiNnb 2023-11-28T10:58:48.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T10:58:48.592 INFO:teuthology.orchestra.run.smithi053.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-28T10:58:48.592 INFO:teuthology.orchestra.run.smithi053.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-28T10:58:48.592 INFO:teuthology.orchestra.run.smithi053.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-28T10:58:48.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-28T10:58:48.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2023-11-28T10:59:14.034 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-28T10:59:14.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T10:59:14.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:18.727 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-28T10:59:18.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T10:59:18.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:32.578 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-28T10:59:37.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-11-28T10:59:41.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T10:59:41.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:44.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-28T10:59:46.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T10:59:46.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:56.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T10:59:56.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:59.721 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-28T11:00:04.578 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-11-28T11:00:14.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T11:00:14.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:16.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T11:00:16.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:18.481 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-11-28T11:00:18.876 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T11:00:18.876 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-28T11:00:18.877 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2023-11-28T11:00:18.919 INFO:teuthology.orchestra.run.smithi053.stdout:6.6.0-g8247bbb5b470 2023-11-28T11:00:18.920 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T11:00:18.920 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:00:18.920 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T11:00:19.920 DEBUG:teuthology.parallel:result is None 2023-11-28T11:00:33.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-28T11:00:51.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T11:00:51.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:51.636 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-11-28T11:00:51.990 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T11:00:51.990 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-11-28T11:00:51.990 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-11-28T11:00:52.032 INFO:teuthology.orchestra.run.smithi157.stdout:6.6.0-g8247bbb5b470 2023-11-28T11:00:52.033 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T11:00:52.033 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:00:52.033 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T11:00:53.034 DEBUG:teuthology.parallel:result is None 2023-11-28T11:00:53.035 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T11:00:53.042 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T11:00:53.042 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:00:53.048 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:00:53.096 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T11:00:53.103 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T11:00:53.116 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T11:00:53.116 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:00:53.121 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:00:53.200 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T11:00:53.207 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T11:00:53.208 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:00:53.212 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:00:53.354 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:00:53.361 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:00:53.390 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:00:53.397 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:00:53.403 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T11:00:53.410 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T11:00:53.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:00:53.415 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:00:53.484 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T11:00:53.495 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T11:00:53.496 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:00:53.501 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:00:53.544 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:00:53.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:00:53.773 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:00:53.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:00:53.929 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:00:53.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:00:54.015 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:00:54.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:00:54.157 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:00:54.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:00:54.297 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:00:54.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:00:54.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2023-11-28T11:00:54.370 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-11-28T11:00:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:00:54.520 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:00:54.956 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T11:00:54.964 INFO:teuthology.task.internal:Starting timer... 2023-11-28T11:00:54.964 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T11:00:54.974 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T11:00:54.984 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T11:00:54.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2023-11-28T11:00:55.099 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2023-11-28T11:00:55.102 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-11-28T11:00:55.221 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2023-11-28T11:00:55.224 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T11:00:55.224 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2023-11-28T11:00:55.278 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2023-11-28T11:00:55.278 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-11-28T11:00:55.339 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-11-28T11:00:55.339 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2023-11-28T11:00:55.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:00:55.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:00:55.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:00:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:00:55.481 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T11:00:55.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:00:55.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:00:55.619 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T11:00:55.629 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T11:00:55.629 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T11:00:55.648 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T11:00:55.649 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:13:46.004 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2023-11-28T11:13:46.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T11:13:46.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:13:46.095 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-11-28T11:13:46.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-28T11:13:46.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T11:13:46.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:13:46.301 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-11-28T11:13:46.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-28T11:13:46.370 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:13:46.379 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:13:46.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:13:46.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:46.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:13:46.385 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:46.457 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:13:46.484 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:13:46.491 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:13:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2023-11-28T11:13:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-11-28T11:13:46.615 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2023-11-28T11:13:46.639 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-11-28T11:13:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:13:46.673 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:13:46.685 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:13:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:13:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.792 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.803 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-11-28T11:13:46.811 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:46.811 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-11-28T11:13:46.811 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.811 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.812 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.812 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:46.813 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:13:46.821 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:13:46.821 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:13:46.821 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:13:46.822 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:13:46.828 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:13:46.828 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:13:46.828 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:13:46.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:13:46.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:13:46.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:13:46.967 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:13:46.967 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:13:46.974 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:13:46.974 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:13:47.074 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:13:47.075 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:13:47.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:13:47.140 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:13:47.140 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:13:47.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:13:47.146 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:13:47.147 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:13:47.210 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:13:47.210 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:13:47.258 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:13:47.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:13:47.355 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:13:47.417 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:13:47.425 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2023-11-28T11:13:47.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2023-11-28T11:13:47.486 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-11-28T11:13:47.489 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-11-28T11:13:47.908 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-28T11:13:47.941 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-11-28T11:13:48.254 INFO:teuthology.orchestra.run.smithi053.stdout:44 files removed 2023-11-28T11:13:48.266 INFO:teuthology.orchestra.run.smithi157.stdout:44 files removed 2023-11-28T11:13:48.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:13:48.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:13:48.783 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-11-28T11:13:48.790 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-28T11:13:49.512 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 299 kB/s | 80 kB 00:00 2023-11-28T11:13:49.580 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 222 kB/s | 80 kB 00:00 2023-11-28T11:13:49.871 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-11-28T11:13:49.880 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-11-28T11:13:50.164 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2023-11-28T11:13:50.198 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-11-28T11:13:50.314 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T11:13:50.332 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-11-28T11:13:51.541 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-11-28T11:13:55.808 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 356 kB/s | 24 kB 00:00 2023-11-28T11:13:56.943 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 47 MB 00:01 2023-11-28T11:14:05.623 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-11-28T11:14:17.161 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:14:17.161 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:14:17.177 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:14:17.178 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:14:17.179 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:14:17.180 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:14:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:14:17.186 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:14:17.188 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:14:17.326 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:14:17.339 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:14:17.340 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:14:17.341 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:14:17.342 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:14:17.343 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:14:17.344 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:14:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:14:17.346 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:14:17.347 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:14:17.348 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:14:17.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:14:17.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:14:17.351 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql 10 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:=========================================================================================================================== 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:Install 128 Packages 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 3 Packages 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:14:17.352 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 203 M 2023-11-28T11:14:17.353 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-11-28T11:14:17.894 INFO:teuthology.orchestra.run.smithi157.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-11-28T11:14:18.236 INFO:teuthology.orchestra.run.smithi157.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 2.4 MB/s | 845 kB 00:00 2023-11-28T11:14:18.320 INFO:teuthology.orchestra.run.smithi157.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.9 MB/s | 165 kB 00:00 2023-11-28T11:14:18.445 INFO:teuthology.orchestra.run.smithi157.stdout:(4/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 7.3 MB/s | 5.4 MB 00:00 2023-11-28T11:14:18.628 INFO:teuthology.orchestra.run.smithi157.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 8.4 MB/s | 1.5 MB 00:00 2023-11-28T11:14:18.945 INFO:teuthology.orchestra.run.smithi157.stdout:(6/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 3.6 MB/s | 2.3 MB 00:00 2023-11-28T11:14:19.120 INFO:teuthology.orchestra.run.smithi157.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 9.4 MB/s | 4.6 MB 00:00 2023-11-28T11:14:19.512 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 559 kB/s | 16 MB 00:29 2023-11-28T11:14:19.838 INFO:teuthology.orchestra.run.smithi157.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 10 MB/s | 22 MB 00:02 2023-11-28T11:14:19.888 INFO:teuthology.orchestra.run.smithi157.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 491 kB/s | 24 kB 00:00 2023-11-28T11:14:20.404 INFO:teuthology.orchestra.run.smithi157.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 10 MB/s | 13 MB 00:01 2023-11-28T11:14:20.480 INFO:teuthology.orchestra.run.smithi157.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 3.4 MB/s | 257 kB 00:00 2023-11-28T11:14:20.547 INFO:teuthology.orchestra.run.smithi157.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 459 kB/s | 30 kB 00:00 2023-11-28T11:14:20.767 INFO:teuthology.orchestra.run.smithi157.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 3.4 MB/s | 767 kB 00:00 2023-11-28T11:14:20.839 INFO:teuthology.orchestra.run.smithi157.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.9 MB/s | 137 kB 00:00 2023-11-28T11:14:20.906 INFO:teuthology.orchestra.run.smithi157.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 2.0 MB/s | 139 kB 00:00 2023-11-28T11:14:20.998 INFO:teuthology.orchestra.run.smithi157.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 5.1 MB/s | 474 kB 00:00 2023-11-28T11:14:21.748 INFO:teuthology.orchestra.run.smithi157.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 8.7 MB/s | 6.5 MB 00:00 2023-11-28T11:14:21.815 INFO:teuthology.orchestra.run.smithi157.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 669 kB/s | 45 kB 00:00 2023-11-28T11:14:21.891 INFO:teuthology.orchestra.run.smithi157.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.5 MB/s | 117 kB 00:00 2023-11-28T11:14:21.983 INFO:teuthology.orchestra.run.smithi157.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.0 MB/s | 190 kB 00:00 2023-11-28T11:14:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 4.4 MB/s | 371 kB 00:00 2023-11-28T11:14:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 4.3 MB/s | 364 kB 00:00 2023-11-28T11:14:22.225 INFO:teuthology.orchestra.run.smithi157.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 107 kB 00:00 2023-11-28T11:14:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-28T11:14:22.609 INFO:teuthology.orchestra.run.smithi157.stdout:(25/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 4.8 MB/s | 18 MB 00:03 2023-11-28T11:14:23.076 INFO:teuthology.orchestra.run.smithi157.stdout:(26/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 15 MB/s | 47 MB 00:03 2023-11-28T11:14:23.101 INFO:teuthology.orchestra.run.smithi157.stdout:(27/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 366 kB/s | 180 kB 00:00 2023-11-28T11:14:23.127 INFO:teuthology.orchestra.run.smithi157.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2697.g 464 kB/s | 23 kB 00:00 2023-11-28T11:14:23.194 INFO:teuthology.orchestra.run.smithi157.stdout:(29/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.9 MB/s | 3.4 MB 00:00 2023-11-28T11:14:23.219 INFO:teuthology.orchestra.run.smithi157.stdout:(30/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.0 MB/s | 123 kB 00:00 2023-11-28T11:14:23.302 INFO:teuthology.orchestra.run.smithi157.stdout:(31/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 9.9 MB/s | 1.7 MB 00:00 2023-11-28T11:14:23.327 INFO:teuthology.orchestra.run.smithi157.stdout:(32/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.1 MB/s | 236 kB 00:00 2023-11-28T11:14:23.361 INFO:teuthology.orchestra.run.smithi157.stdout:(33/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 812 kB/s | 47 kB 00:00 2023-11-28T11:14:23.395 INFO:teuthology.orchestra.run.smithi157.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2697.g2 217 kB/s | 15 kB 00:00 2023-11-28T11:14:23.429 INFO:teuthology.orchestra.run.smithi157.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 3.8 MB/s | 263 kB 00:00 2023-11-28T11:14:23.470 INFO:teuthology.orchestra.run.smithi157.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.1 MB/s | 83 kB 00:00 2023-11-28T11:14:23.496 INFO:teuthology.orchestra.run.smithi157.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 271 kB/s | 18 kB 00:00 2023-11-28T11:14:23.688 INFO:teuthology.orchestra.run.smithi157.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-11-28T11:14:23.821 INFO:teuthology.orchestra.run.smithi157.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 181 kB/s | 59 kB 00:00 2023-11-28T11:14:23.904 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-11-28T11:14:24.071 INFO:teuthology.orchestra.run.smithi157.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 510 kB/s | 306 kB 00:00 2023-11-28T11:14:24.097 INFO:teuthology.orchestra.run.smithi157.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 184 kB/s | 75 kB 00:00 2023-11-28T11:14:24.164 INFO:teuthology.orchestra.run.smithi157.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-11-28T11:14:24.181 INFO:teuthology.orchestra.run.smithi157.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 400 kB/s | 44 kB 00:00 2023-11-28T11:14:24.264 INFO:teuthology.orchestra.run.smithi157.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2023-11-28T11:14:24.289 INFO:teuthology.orchestra.run.smithi157.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-11-28T11:14:24.323 INFO:teuthology.orchestra.run.smithi157.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 819 kB/s | 185 kB 00:00 2023-11-28T11:14:24.391 INFO:teuthology.orchestra.run.smithi157.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-11-28T11:14:24.441 INFO:teuthology.orchestra.run.smithi157.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 694 kB/s | 81 kB 00:00 2023-11-28T11:14:24.491 INFO:teuthology.orchestra.run.smithi157.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2023-11-28T11:14:24.508 INFO:teuthology.orchestra.run.smithi157.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-11-28T11:14:24.566 INFO:teuthology.orchestra.run.smithi157.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-11-28T11:14:24.633 INFO:teuthology.orchestra.run.smithi157.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-11-28T11:14:24.700 INFO:teuthology.orchestra.run.smithi157.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2023-11-28T11:14:24.776 INFO:teuthology.orchestra.run.smithi157.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2023-11-28T11:14:24.801 INFO:teuthology.orchestra.run.smithi157.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 361 kB/s | 60 kB 00:00 2023-11-28T11:14:24.885 INFO:teuthology.orchestra.run.smithi157.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-11-28T11:14:24.926 INFO:teuthology.orchestra.run.smithi157.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-11-28T11:14:24.952 INFO:teuthology.orchestra.run.smithi157.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-11-28T11:14:25.073 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-11-28T11:14:25.085 INFO:teuthology.orchestra.run.smithi157.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 981 kB/s | 196 kB 00:00 2023-11-28T11:14:25.111 INFO:teuthology.orchestra.run.smithi157.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 118 kB/s | 19 kB 00:00 2023-11-28T11:14:25.136 INFO:teuthology.orchestra.run.smithi157.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 307 kB/s | 64 kB 00:00 2023-11-28T11:14:25.203 INFO:teuthology.orchestra.run.smithi157.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-11-28T11:14:25.245 INFO:teuthology.orchestra.run.smithi157.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-11-28T11:14:25.270 INFO:teuthology.orchestra.run.smithi157.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 206 kB/s | 27 kB 00:00 2023-11-28T11:14:25.420 INFO:teuthology.orchestra.run.smithi157.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 408 kB/s | 61 kB 00:00 2023-11-28T11:14:25.446 INFO:teuthology.orchestra.run.smithi157.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-11-28T11:14:25.586 INFO:teuthology.orchestra.run.smithi157.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 738 kB/s | 251 kB 00:00 2023-11-28T11:14:25.613 INFO:teuthology.orchestra.run.smithi157.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 445 kB/s | 85 kB 00:00 2023-11-28T11:14:25.638 INFO:teuthology.orchestra.run.smithi157.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 119 kB/s | 23 kB 00:00 2023-11-28T11:14:25.672 INFO:teuthology.orchestra.run.smithi157.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2023-11-28T11:14:25.730 INFO:teuthology.orchestra.run.smithi157.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-11-28T11:14:25.764 INFO:teuthology.orchestra.run.smithi157.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-28T11:14:25.806 INFO:teuthology.orchestra.run.smithi157.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-28T11:14:26.073 INFO:teuthology.orchestra.run.smithi157.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2023-11-28T11:14:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-11-28T11:14:26.265 INFO:teuthology.orchestra.run.smithi157.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-11-28T11:14:26.316 INFO:teuthology.orchestra.run.smithi157.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-11-28T11:14:26.358 INFO:teuthology.orchestra.run.smithi157.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-28T11:14:26.400 INFO:teuthology.orchestra.run.smithi157.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-28T11:14:26.450 INFO:teuthology.orchestra.run.smithi157.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-28T11:14:26.608 INFO:teuthology.orchestra.run.smithi157.stdout:(81/131): python3-scipy-1.0.0-21.module+el8.5.0 14 MB/s | 14 MB 00:00 2023-11-28T11:14:26.683 INFO:teuthology.orchestra.run.smithi157.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 356 kB/s | 27 kB 00:00 2023-11-28T11:14:26.734 INFO:teuthology.orchestra.run.smithi157.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-11-28T11:14:26.776 INFO:teuthology.orchestra.run.smithi157.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-11-28T11:14:26.868 INFO:teuthology.orchestra.run.smithi157.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-11-28T11:14:26.910 INFO:teuthology.orchestra.run.smithi157.stdout:(86/131): thrift-0.13.0-2.el8.x86_64.rpm 1.3 MB/s | 1.7 MB 00:01 2023-11-28T11:14:26.952 INFO:teuthology.orchestra.run.smithi157.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 5.3 MB/s | 457 kB 00:00 2023-11-28T11:14:26.985 INFO:teuthology.orchestra.run.smithi157.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.9 MB/s | 225 kB 00:00 2023-11-28T11:14:27.011 INFO:teuthology.orchestra.run.smithi157.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-11-28T11:14:27.036 INFO:teuthology.orchestra.run.smithi157.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-11-28T11:14:27.061 INFO:teuthology.orchestra.run.smithi157.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-11-28T11:14:27.086 INFO:teuthology.orchestra.run.smithi157.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-28T11:14:27.129 INFO:teuthology.orchestra.run.smithi157.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-28T11:14:27.171 INFO:teuthology.orchestra.run.smithi157.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-28T11:14:27.262 INFO:teuthology.orchestra.run.smithi157.stdout:(95/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.7 MB/s | 3.7 MB 00:00 2023-11-28T11:14:27.305 INFO:teuthology.orchestra.run.smithi157.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-28T11:14:27.346 INFO:teuthology.orchestra.run.smithi157.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.4 MB/s | 354 kB 00:00 2023-11-28T11:14:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-11-28T11:14:27.539 INFO:teuthology.orchestra.run.smithi157.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-11-28T11:14:27.581 INFO:teuthology.orchestra.run.smithi157.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-11-28T11:14:27.614 INFO:teuthology.orchestra.run.smithi157.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-11-28T11:14:27.665 INFO:teuthology.orchestra.run.smithi157.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-28T11:14:27.690 INFO:teuthology.orchestra.run.smithi157.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-11-28T11:14:27.707 INFO:teuthology.orchestra.run.smithi157.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-11-28T11:14:27.771 INFO:teuthology.orchestra.run.smithi157.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.7 MB/s | 220 kB 00:00 2023-11-28T11:14:27.799 INFO:teuthology.orchestra.run.smithi157.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 524 kB/s | 48 kB 00:00 2023-11-28T11:14:27.833 INFO:teuthology.orchestra.run.smithi157.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-11-28T11:14:27.858 INFO:teuthology.orchestra.run.smithi157.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 728 kB/s | 43 kB 00:00 2023-11-28T11:14:27.891 INFO:teuthology.orchestra.run.smithi157.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2023-11-28T11:14:27.925 INFO:teuthology.orchestra.run.smithi157.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-11-28T11:14:27.992 INFO:teuthology.orchestra.run.smithi157.stdout:(111/131): postgresql-server-10.21-2.module+el8 7.8 MB/s | 5.1 MB 00:00 2023-11-28T11:14:28.017 INFO:teuthology.orchestra.run.smithi157.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-11-28T11:14:28.042 INFO:teuthology.orchestra.run.smithi157.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-28T11:14:28.076 INFO:teuthology.orchestra.run.smithi157.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-11-28T11:14:28.101 INFO:teuthology.orchestra.run.smithi157.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-11-28T11:14:28.135 INFO:teuthology.orchestra.run.smithi157.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-11-28T11:14:28.169 INFO:teuthology.orchestra.run.smithi157.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-28T11:14:28.202 INFO:teuthology.orchestra.run.smithi157.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-28T11:14:28.227 INFO:teuthology.orchestra.run.smithi157.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-11-28T11:14:28.245 INFO:teuthology.orchestra.run.smithi157.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-28T11:14:28.270 INFO:teuthology.orchestra.run.smithi157.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-11-28T11:14:28.295 INFO:teuthology.orchestra.run.smithi157.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-28T11:14:28.345 INFO:teuthology.orchestra.run.smithi157.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 768 kB/s | 58 kB 00:00 2023-11-28T11:14:28.371 INFO:teuthology.orchestra.run.smithi157.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 512 kB/s | 39 kB 00:00 2023-11-28T11:14:28.421 INFO:teuthology.orchestra.run.smithi157.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-11-28T11:14:28.446 INFO:teuthology.orchestra.run.smithi157.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-11-28T11:14:28.471 INFO:teuthology.orchestra.run.smithi157.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-28T11:14:28.530 INFO:teuthology.orchestra.run.smithi157.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.9 MB/s | 2.3 MB 00:00 2023-11-28T11:14:28.580 INFO:teuthology.orchestra.run.smithi157.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T11:14:28.781 INFO:teuthology.orchestra.run.smithi157.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 12 MB/s | 3.6 MB 00:00 2023-11-28T11:14:29.105 INFO:teuthology.orchestra.run.smithi157.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 5.5 MB/s | 3.7 MB 00:00 2023-11-28T11:14:29.108 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:14:29.108 INFO:teuthology.orchestra.run.smithi157.stdout:Total 17 MB/s | 203 MB 00:11 2023-11-28T11:14:30.790 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-11-28T11:14:30.928 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-11-28T11:14:30.981 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-11-28T11:14:32.671 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-11-28T11:14:32.676 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-11-28T11:14:34.190 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-11-28T11:14:37.997 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-11-28T11:14:38.188 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:14:38.365 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:14:38.632 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:14:38.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:14:38.991 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:14:39.189 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:14:39.656 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:14:39.896 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:14:40.363 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:14:40.572 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:14:40.729 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:14:40.913 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:14:41.034 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:14:41.543 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:14:41.832 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:14:42.069 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:14:42.556 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:14:42.809 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:14:43.046 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:14:43.215 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:14:43.421 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:14:43.527 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:14:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:14:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:14:44.013 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:14:44.702 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:14:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:14:45.550 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:14:45.746 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:14:46.598 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:14:46.667 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:14:46.668 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:14:46.684 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:14:46.685 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:14:46.686 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:14:46.687 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:14:46.690 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:14:46.693 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:14:46.694 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:14:46.844 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:14:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:14:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:14:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2023-11-28T11:14:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:14:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:14:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:14:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:14:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:14:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:14:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:14:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:14:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql 10 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout:Install 128 Packages 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 3 Packages 2023-11-28T11:14:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:14:46.871 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 203 M 2023-11-28T11:14:46.871 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-11-28T11:14:47.063 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:14:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-11-28T11:14:47.300 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:14:47.536 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:14:47.682 INFO:teuthology.orchestra.run.smithi053.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.4 MB/s | 845 kB 00:00 2023-11-28T11:14:47.746 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:14:47.841 INFO:teuthology.orchestra.run.smithi053.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.0 MB/s | 165 kB 00:00 2023-11-28T11:14:47.973 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:14:48.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:14:48.349 INFO:teuthology.orchestra.run.smithi053.stdout:(4/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 15 MB/s | 22 MB 00:01 2023-11-28T11:14:48.453 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:14:48.466 INFO:teuthology.orchestra.run.smithi053.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 13 MB/s | 1.5 MB 00:00 2023-11-28T11:14:48.575 INFO:teuthology.orchestra.run.smithi053.stdout:(6/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 3.2 MB/s | 5.4 MB 00:01 2023-11-28T11:14:48.591 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:14:48.709 INFO:teuthology.orchestra.run.smithi053.stdout:(7/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-11-28T11:14:48.792 INFO:teuthology.orchestra.run.smithi053.stdout:(8/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 14 MB/s | 4.6 MB 00:00 2023-11-28T11:14:48.842 INFO:teuthology.orchestra.run.smithi053.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 490 kB/s | 24 kB 00:00 2023-11-28T11:14:49.695 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:14:50.069 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:14:50.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:14:50.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:14:50.717 INFO:teuthology.orchestra.run.smithi053.stdout:(10/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 8.3 MB/s | 18 MB 00:02 2023-11-28T11:14:50.793 INFO:teuthology.orchestra.run.smithi053.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 3.4 MB/s | 257 kB 00:00 2023-11-28T11:14:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:14:50.868 INFO:teuthology.orchestra.run.smithi053.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 406 kB/s | 30 kB 00:00 2023-11-28T11:14:50.994 INFO:teuthology.orchestra.run.smithi053.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 6.0 MB/s | 767 kB 00:00 2023-11-28T11:14:51.086 INFO:teuthology.orchestra.run.smithi053.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 137 kB 00:00 2023-11-28T11:14:51.161 INFO:teuthology.orchestra.run.smithi053.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.8 MB/s | 139 kB 00:00 2023-11-28T11:14:51.261 INFO:teuthology.orchestra.run.smithi053.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 4.6 MB/s | 474 kB 00:00 2023-11-28T11:14:52.003 INFO:teuthology.orchestra.run.smithi053.stdout:(17/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 15 MB/s | 47 MB 00:03 2023-11-28T11:14:52.187 INFO:teuthology.orchestra.run.smithi053.stdout:(18/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 3.7 MB/s | 13 MB 00:03 2023-11-28T11:14:52.212 INFO:teuthology.orchestra.run.smithi053.stdout:(19/131): python3-ceph-argparse-18.0.0-2697.g2f 214 kB/s | 45 kB 00:00 2023-11-28T11:14:52.304 INFO:teuthology.orchestra.run.smithi053.stdout:(20/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 6.2 MB/s | 6.5 MB 00:01 2023-11-28T11:14:52.329 INFO:teuthology.orchestra.run.smithi053.stdout:(21/131): python3-ceph-common-18.0.0-2697.g2f5f 824 kB/s | 117 kB 00:00 2023-11-28T11:14:52.354 INFO:teuthology.orchestra.run.smithi053.stdout:(22/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.3 MB/s | 190 kB 00:00 2023-11-28T11:14:52.380 INFO:teuthology.orchestra.run.smithi053.stdout:(23/131): python3-rados-18.0.0-2697.g2f5f6028.e 4.8 MB/s | 371 kB 00:00 2023-11-28T11:14:52.405 INFO:teuthology.orchestra.run.smithi053.stdout:(24/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 107 kB 00:00 2023-11-28T11:14:52.447 INFO:teuthology.orchestra.run.smithi053.stdout:(25/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 3.0 MB/s | 364 kB 00:00 2023-11-28T11:14:52.472 INFO:teuthology.orchestra.run.smithi053.stdout:(26/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.0 MB/s | 94 kB 00:00 2023-11-28T11:14:52.533 INFO:teuthology.orchestra.run.smithi053.stdout:(27/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.1 MB/s | 180 kB 00:00 2023-11-28T11:14:52.548 INFO:teuthology.orchestra.run.smithi053.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2697.g 306 kB/s | 23 kB 00:00 2023-11-28T11:14:52.823 INFO:teuthology.orchestra.run.smithi053.stdout:(29/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 8.3 MB/s | 3.4 MB 00:00 2023-11-28T11:14:52.872 INFO:teuthology.orchestra.run.smithi053.stdout:(30/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 360 kB/s | 123 kB 00:00 2023-11-28T11:14:52.922 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:14:52.932 INFO:teuthology.orchestra.run.smithi053.stdout:(31/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.5 MB/s | 1.7 MB 00:00 2023-11-28T11:14:52.991 INFO:teuthology.orchestra.run.smithi053.stdout:(32/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.0 MB/s | 236 kB 00:00 2023-11-28T11:14:53.016 INFO:teuthology.orchestra.run.smithi053.stdout:(33/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 568 kB/s | 47 kB 00:00 2023-11-28T11:14:53.067 INFO:teuthology.orchestra.run.smithi053.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2697.g2 193 kB/s | 15 kB 00:00 2023-11-28T11:14:53.117 INFO:teuthology.orchestra.run.smithi053.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.6 MB/s | 263 kB 00:00 2023-11-28T11:14:53.142 INFO:teuthology.orchestra.run.smithi053.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.1 MB/s | 83 kB 00:00 2023-11-28T11:14:53.192 INFO:teuthology.orchestra.run.smithi053.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 240 kB/s | 18 kB 00:00 2023-11-28T11:14:53.284 INFO:teuthology.orchestra.run.smithi053.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 16 MB/s | 7.4 MB 00:00 2023-11-28T11:14:53.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:14:53.368 INFO:teuthology.orchestra.run.smithi053.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2023-11-28T11:14:53.402 INFO:teuthology.orchestra.run.smithi053.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-28T11:14:53.427 INFO:teuthology.orchestra.run.smithi053.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 528 kB/s | 75 kB 00:00 2023-11-28T11:14:53.452 INFO:teuthology.orchestra.run.smithi053.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 872 kB/s | 44 kB 00:00 2023-11-28T11:14:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-11-28T11:14:53.520 INFO:teuthology.orchestra.run.smithi053.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 493 kB/s | 33 kB 00:00 2023-11-28T11:14:53.537 INFO:teuthology.orchestra.run.smithi053.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2023-11-28T11:14:53.549 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:14:53.595 INFO:teuthology.orchestra.run.smithi053.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-11-28T11:14:53.620 INFO:teuthology.orchestra.run.smithi053.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-11-28T11:14:53.646 INFO:teuthology.orchestra.run.smithi053.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-28T11:14:53.663 INFO:teuthology.orchestra.run.smithi053.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-11-28T11:14:53.680 INFO:teuthology.orchestra.run.smithi053.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 529 kB/s | 18 kB 00:00 2023-11-28T11:14:53.705 INFO:teuthology.orchestra.run.smithi053.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-11-28T11:14:53.730 INFO:teuthology.orchestra.run.smithi053.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 971 kB/s | 49 kB 00:00 2023-11-28T11:14:53.758 INFO:teuthology.orchestra.run.smithi053.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-11-28T11:14:53.767 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:14:53.781 INFO:teuthology.orchestra.run.smithi053.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-28T11:14:53.806 INFO:teuthology.orchestra.run.smithi053.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-28T11:14:53.831 INFO:teuthology.orchestra.run.smithi053.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-11-28T11:14:53.857 INFO:teuthology.orchestra.run.smithi053.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 673 kB/s | 34 kB 00:00 2023-11-28T11:14:53.882 INFO:teuthology.orchestra.run.smithi053.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-28T11:14:53.907 INFO:teuthology.orchestra.run.smithi053.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-11-28T11:14:53.933 INFO:teuthology.orchestra.run.smithi053.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-11-28T11:14:53.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:14:53.958 INFO:teuthology.orchestra.run.smithi053.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-11-28T11:14:53.983 INFO:teuthology.orchestra.run.smithi053.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-11-28T11:14:54.008 INFO:teuthology.orchestra.run.smithi053.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-11-28T11:14:54.050 INFO:teuthology.orchestra.run.smithi053.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 3.0 MB/s | 1.2 MB 00:00 2023-11-28T11:14:54.084 INFO:teuthology.orchestra.run.smithi053.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-11-28T11:14:54.118 INFO:teuthology.orchestra.run.smithi053.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 917 kB/s | 61 kB 00:00 2023-11-28T11:14:54.152 INFO:teuthology.orchestra.run.smithi053.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-11-28T11:14:54.166 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:14:54.177 INFO:teuthology.orchestra.run.smithi053.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-28T11:14:54.202 INFO:teuthology.orchestra.run.smithi053.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-11-28T11:14:54.227 INFO:teuthology.orchestra.run.smithi053.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-28T11:14:54.294 INFO:teuthology.orchestra.run.smithi053.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 512 kB/s | 47 kB 00:00 2023-11-28T11:14:54.328 INFO:teuthology.orchestra.run.smithi053.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-11-28T11:14:54.330 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:14:54.353 INFO:teuthology.orchestra.run.smithi053.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 729 kB/s | 43 kB 00:00 2023-11-28T11:14:54.387 INFO:teuthology.orchestra.run.smithi053.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-11-28T11:14:54.404 INFO:teuthology.orchestra.run.smithi053.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2023-11-28T11:14:54.446 INFO:teuthology.orchestra.run.smithi053.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2023-11-28T11:14:54.471 INFO:teuthology.orchestra.run.smithi053.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2023-11-28T11:14:54.505 INFO:teuthology.orchestra.run.smithi053.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 427 kB/s | 43 kB 00:00 2023-11-28T11:14:54.529 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:14:54.530 INFO:teuthology.orchestra.run.smithi053.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2023-11-28T11:14:54.672 INFO:teuthology.orchestra.run.smithi053.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-11-28T11:14:54.697 INFO:teuthology.orchestra.run.smithi053.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 112 kB/s | 19 kB 00:00 2023-11-28T11:14:54.731 INFO:teuthology.orchestra.run.smithi053.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-11-28T11:14:54.756 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:14:54.765 INFO:teuthology.orchestra.run.smithi053.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.1 MB/s | 560 kB 00:00 2023-11-28T11:14:54.790 INFO:teuthology.orchestra.run.smithi053.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-11-28T11:14:54.832 INFO:teuthology.orchestra.run.smithi053.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-11-28T11:14:54.865 INFO:teuthology.orchestra.run.smithi053.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-11-28T11:14:54.899 INFO:teuthology.orchestra.run.smithi053.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-11-28T11:14:54.908 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:14:54.924 INFO:teuthology.orchestra.run.smithi053.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-11-28T11:14:54.950 INFO:teuthology.orchestra.run.smithi053.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T11:14:55.000 INFO:teuthology.orchestra.run.smithi053.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 799 kB/s | 60 kB 00:00 2023-11-28T11:14:55.025 INFO:teuthology.orchestra.run.smithi053.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2023-11-28T11:14:55.051 INFO:teuthology.orchestra.run.smithi053.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-28T11:14:55.076 INFO:teuthology.orchestra.run.smithi053.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-11-28T11:14:55.118 INFO:teuthology.orchestra.run.smithi053.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.1 MB/s | 171 kB 00:00 2023-11-28T11:14:55.125 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:14:55.151 INFO:teuthology.orchestra.run.smithi053.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.3 MB/s | 644 kB 00:00 2023-11-28T11:14:55.194 INFO:teuthology.orchestra.run.smithi053.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.8 MB/s | 138 kB 00:00 2023-11-28T11:14:55.236 INFO:teuthology.orchestra.run.smithi053.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-28T11:14:55.278 INFO:teuthology.orchestra.run.smithi053.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-28T11:14:55.328 INFO:teuthology.orchestra.run.smithi053.stdout:(99/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 4.7 MB/s | 2.3 MB 00:00 2023-11-28T11:14:55.362 INFO:teuthology.orchestra.run.smithi053.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-11-28T11:14:55.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:14:55.395 INFO:teuthology.orchestra.run.smithi053.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-11-28T11:14:55.421 INFO:teuthology.orchestra.run.smithi053.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-11-28T11:14:55.471 INFO:teuthology.orchestra.run.smithi053.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-11-28T11:14:55.513 INFO:teuthology.orchestra.run.smithi053.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.7 MB/s | 538 kB 00:00 2023-11-28T11:14:55.546 INFO:teuthology.orchestra.run.smithi053.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-11-28T11:14:55.580 INFO:teuthology.orchestra.run.smithi053.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-11-28T11:14:55.630 INFO:teuthology.orchestra.run.smithi053.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-11-28T11:14:55.700 INFO:teuthology.orchestra.run.smithi053.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 384 kB/s | 27 kB 00:00 2023-11-28T11:14:55.756 INFO:teuthology.orchestra.run.smithi053.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 5.6 MB/s | 320 kB 00:00 2023-11-28T11:14:55.789 INFO:teuthology.orchestra.run.smithi053.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-28T11:14:55.801 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:14:55.940 INFO:teuthology.orchestra.run.smithi053.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-11-28T11:14:55.998 INFO:teuthology.orchestra.run.smithi053.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-11-28T11:14:56.065 INFO:teuthology.orchestra.run.smithi053.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-11-28T11:14:56.099 INFO:teuthology.orchestra.run.smithi053.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-11-28T11:14:56.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:14:56.124 INFO:teuthology.orchestra.run.smithi053.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-11-28T11:14:56.149 INFO:teuthology.orchestra.run.smithi053.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-28T11:14:56.175 INFO:teuthology.orchestra.run.smithi053.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-11-28T11:14:56.200 INFO:teuthology.orchestra.run.smithi053.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-28T11:14:56.242 INFO:teuthology.orchestra.run.smithi053.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-28T11:14:56.292 INFO:teuthology.orchestra.run.smithi053.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.8 MB/s | 198 kB 00:00 2023-11-28T11:14:56.334 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:14:56.493 INFO:teuthology.orchestra.run.smithi053.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 10 MB/s | 14 MB 00:01 2023-11-28T11:14:56.535 INFO:teuthology.orchestra.run.smithi053.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-28T11:14:56.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:14:56.585 INFO:teuthology.orchestra.run.smithi053.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-11-28T11:14:56.677 INFO:teuthology.orchestra.run.smithi053.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 10 MB/s | 4.9 MB 00:00 2023-11-28T11:14:56.777 INFO:teuthology.orchestra.run.smithi053.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2023-11-28T11:14:56.805 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:14:56.819 INFO:teuthology.orchestra.run.smithi053.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-11-28T11:14:56.895 INFO:teuthology.orchestra.run.smithi053.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-11-28T11:14:56.995 INFO:teuthology.orchestra.run.smithi053.stdout:(128/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-11-28T11:14:57.053 INFO:teuthology.orchestra.run.smithi053.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-28T11:14:57.165 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:14:57.421 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:14:57.612 INFO:teuthology.orchestra.run.smithi053.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 5.0 MB/s | 3.6 MB 00:00 2023-11-28T11:14:57.635 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:14:57.872 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:14:57.946 INFO:teuthology.orchestra.run.smithi053.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 3.2 MB/s | 3.7 MB 00:01 2023-11-28T11:14:57.949 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:14:57.949 INFO:teuthology.orchestra.run.smithi053.stdout:Total 18 MB/s | 203 MB 00:11 2023-11-28T11:14:58.120 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:14:58.291 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:14:58.661 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:14:58.950 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:14:59.226 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:14:59.469 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:14:59.651 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-11-28T11:14:59.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:14:59.798 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-11-28T11:14:59.798 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-11-28T11:14:59.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:15:00.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:15:00.127 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:15:00.415 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:15:00.641 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:15:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:15:00.943 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:15:01.179 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:15:01.404 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:15:01.580 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-11-28T11:15:01.586 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:15:01.588 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-11-28T11:15:01.802 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:15:02.112 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:15:02.377 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:15:02.565 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:15:02.689 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:15:02.883 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:15:03.126 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:15:03.325 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:15:03.488 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:15:03.671 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:15:03.920 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:15:04.149 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:15:04.376 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:15:04.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:15:04.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:15:05.097 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:15:05.296 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:15:06.645 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:15:06.800 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-11-28T11:15:06.931 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:15:07.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:15:07.207 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:15:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:15:07.520 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:15:07.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:15:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:15:07.852 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:15:08.075 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:15:08.246 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:15:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:15:08.966 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:15:08.988 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:15:09.166 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:15:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:15:09.241 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:15:09.327 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:15:09.468 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:15:09.510 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:15:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:15:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:15:10.016 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T11:15:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:15:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:15:11.010 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:15:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:15:11.477 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:15:11.590 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:15:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:15:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:15:12.226 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T11:15:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T11:15:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T11:15:12.751 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:15:12.919 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:15:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:15:14.112 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:15:14.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:15:14.414 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:14.642 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:15:15.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:15:15.315 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:15.659 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:15:15.804 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:15:15.982 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:15:16.079 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:15:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T11:15:16.573 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T11:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:15:17.034 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:15:17.172 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:15:17.407 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:15:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:15:19.792 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:15:20.982 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T11:15:21.362 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T11:15:21.603 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T11:15:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T11:15:22.048 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:15:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:15:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:15:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T11:15:22.979 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T11:15:23.150 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:15:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:15:23.852 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T11:15:24.090 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T11:15:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:15:24.361 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:15:24.571 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T11:15:24.780 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T11:15:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T11:15:25.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T11:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T11:15:25.516 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T11:15:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T11:15:25.944 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T11:15:26.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T11:15:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T11:15:26.891 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T11:15:27.168 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T11:15:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T11:15:27.768 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T11:15:27.993 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T11:15:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T11:15:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T11:15:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T11:15:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T11:15:29.154 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T11:15:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T11:15:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T11:15:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T11:15:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:15:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:15:30.596 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:15:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:15:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:15:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:15:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:15:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:15:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:15:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:15:32.337 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:15:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:15:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:15:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:15:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:15:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:15:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:15:35.125 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:15:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:15:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:15:35.954 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:15:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:15:36.437 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:15:36.457 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:15:36.457 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-11-28T11:15:36.457 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-11-28T11:15:36.457 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-11-28T11:15:36.458 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:15:36.879 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:15:37.498 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:15:37.592 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:15:37.757 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:15:38.595 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:15:38.796 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:15:39.062 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:15:39.238 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:15:39.764 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:15:40.054 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:15:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:15:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:15:43.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:15:43.212 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:43.286 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:15:43.406 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:15:43.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:15:43.933 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:15:44.178 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:15:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:15:44.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:15:44.922 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:15:45.280 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:15:45.494 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:15:45.623 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:15:46.019 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:15:46.220 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:15:46.286 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:15:46.494 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:15:46.524 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:15:47.859 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:15:48.125 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:15:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:15:48.352 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:15:48.423 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:15:48.586 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:15:48.990 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:15:49.202 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:15:54.065 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:15:54.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:15:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:15:54.848 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:15:56.155 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:15:56.515 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:15:56.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:15:56.947 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:15:57.156 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:15:57.293 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:16:00.743 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:01.009 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:01.049 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:01.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:16:01.560 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:16:01.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:16:02.202 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:16:02.441 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:16:02.465 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:16:02.679 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:16:02.883 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:16:02.999 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:16:03.233 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:16:03.262 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:16:03.262 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:03.522 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:03.523 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:16:03.790 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:16:04.020 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:04.296 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:04.296 INFO:teuthology.orchestra.run.smithi157.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T11:16:04.296 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:16:04.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:16:04.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:16:04.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:16:04.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:16:04.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:16:04.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:16:04.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:16:04.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:16:04.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:16:04.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:16:04.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:16:04.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:16:04.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:16:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:16:04.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:16:04.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:16:04.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:04.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2023-11-28T11:16:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2023-11-28T11:16:06.070 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2023-11-28T11:16:06.070 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2023-11-28T11:16:06.070 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-28T11:16:06.070 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:16:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:07.679 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:02 ago on Tue 28 Nov 2023 11:14:05 AM UTC. 2023-11-28T11:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:16:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:16:08.736 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:16:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:16:09.436 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:16:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:16:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:16:10.300 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-11-28T11:16:10.564 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:10.564 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-11-28T11:16:10.564 INFO:teuthology.orchestra.run.smithi157.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:16:10.564 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:16:10.565 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.566 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.567 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.568 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:16:10.569 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.570 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.571 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.571 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:16:10.571 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:16:10.571 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:16:10.572 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:10.573 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:16:10.574 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:16:10.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:16:10.576 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:16:10.577 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:16:10.578 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:16:10.579 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:16:10.580 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:10.580 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-11-28T11:16:11.069 DEBUG:teuthology.parallel:result is None 2023-11-28T11:16:12.780 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:16:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:16:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:16:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:16:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:16:13.891 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:16:14.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:16:15.228 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:16:15.393 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:16:15.525 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:16:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:16:16.139 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:16:16.206 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:16:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:16:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:16:18.961 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:16:19.268 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:16:24.380 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:16:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:16:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T11:16:26.391 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T11:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T11:16:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:16:27.056 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:16:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T11:16:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:16:27.593 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:16:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:31.571 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:16:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:16:32.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:16:32.355 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:16:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:16:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:16:33.400 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:16:33.428 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:16:33.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:16:33.623 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:16:33.795 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T11:16:33.795 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:34.023 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:34.023 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:16:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:16:34.492 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:34.774 INFO:teuthology.orchestra.run.smithi053.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T11:16:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:35.412 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:16:35.413 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:16:35.414 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:16:35.415 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:16:35.415 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:16:35.415 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:16:35.424 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:16:35.424 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:16:35.424 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:16:35.424 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:16:35.424 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:16:35.425 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:16:35.426 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:16:35.427 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T11:16:35.428 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T11:16:35.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T11:16:35.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T11:16:35.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T11:16:35.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:16:35.433 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:16:35.433 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:16:35.433 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:16:35.433 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:16:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:16:38.183 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:05 ago on Tue 28 Nov 2023 11:14:33 AM UTC. 2023-11-28T11:16:40.925 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2023-11-28T11:16:41.174 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.176 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.177 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:16:41.178 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.179 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:16:41.180 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:16:41.182 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:16:41.183 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:16:41.184 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:16:41.185 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:16:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:16:41.187 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:16:41.187 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:16:41.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:41.188 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-28T11:16:41.458 DEBUG:teuthology.parallel:result is None 2023-11-28T11:16:41.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:16:41.606 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:16:41.662 INFO:teuthology.orchestra.run.smithi053.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:16:41.663 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:16:41.663 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:16:41.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:16:41.802 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:16:41.859 INFO:teuthology.orchestra.run.smithi157.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:16:41.859 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:16:41.859 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:16:41.860 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:16:41.860 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:41.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:16:41.953 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:41.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:16:42.046 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:16:42.046 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:42.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:16:42.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:16:42.233 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:42.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:16:42.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:16:42.416 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:16:42.416 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:42.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:16:42.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:16:42.599 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:42.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:16:42.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:16:42.781 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:16:42.781 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:42.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:16:42.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:16:42.963 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:42.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:16:43.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:16:43.144 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:16:43.206 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:16:43.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2023-11-28T11:16:43.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2023-11-28T11:16:43.280 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:16:43.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:16:43.285 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:16:43.356 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:16:43.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:16:43.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:16:43.438 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:16:43.438 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:16:43.438 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:16:43.438 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:43.439 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:16:43.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:16:43.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:16:43.764 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:43.765 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:16:43.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:16:44.028 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:16:44.103 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:16:44.103 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:16:44.104 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469958', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9437, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_10:48:03', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:16:44.104 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:16:44.108 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:16:44.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:16:44.194 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:16:44.266 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:44.266 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:16:44.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:16:44.310 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 451 Links: 1 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:44.396 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-28 11:16:34.900774461 +0000 2023-11-28T11:16:44.397 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-28 11:13:05.146776628 +0000 2023-11-28T11:16:44.397 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-28 11:13:05.146776628 +0000 2023-11-28T11:16:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-11-28 11:13:05.124777043 +0000 2023-11-28T11:16:44.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:16:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-28T11:16:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-28T11:16:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000232694 s, 2.2 MB/s 2023-11-28T11:16:44.510 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:16:44.558 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2023-11-28T11:16:44.603 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:16:44.603 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:44.603 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 457 Links: 1 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-28 11:16:34.900774461 +0000 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-28 11:13:05.979760924 +0000 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-28 11:13:05.979760924 +0000 2023-11-28T11:16:44.604 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-11-28 11:13:05.957761339 +0000 2023-11-28T11:16:44.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:16:44.713 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-28T11:16:44.713 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-28T11:16:44.713 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000188143 s, 2.7 MB/s 2023-11-28T11:16:44.716 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:16:44.762 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2023-11-28T11:16:44.848 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:16:44.848 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:44.848 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 463 Links: 1 2023-11-28T11:16:44.848 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:44.848 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:44.849 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-28 11:16:34.900774461 +0000 2023-11-28T11:16:44.849 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-28 11:13:06.813745202 +0000 2023-11-28T11:16:44.849 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-28 11:13:06.813745202 +0000 2023-11-28T11:16:44.849 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-11-28 11:13:06.791745616 +0000 2023-11-28T11:16:44.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:16:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-28T11:16:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-28T11:16:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000186676 s, 2.7 MB/s 2023-11-28T11:16:44.956 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:16:45.002 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2023-11-28T11:16:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 469 Links: 1 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-28 11:16:34.900774461 +0000 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-28 11:13:07.637729668 +0000 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-28 11:13:07.637729668 +0000 2023-11-28T11:16:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-11-28 11:13:07.614730101 +0000 2023-11-28T11:16:45.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:16:45.190 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-28T11:16:45.190 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-28T11:16:45.191 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000201639 s, 2.5 MB/s 2023-11-28T11:16:45.193 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:16:45.238 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:45.238 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:45.238 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:16:45.283 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:16:45.283 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-11-28T11:16:45.368 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:16:45.368 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-28 11:16:04.424395975 +0000 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-28 11:12:45.192304459 +0000 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-28 11:12:45.192304459 +0000 2023-11-28T11:16:45.369 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-11-28 11:12:45.169304854 +0000 2023-11-28T11:16:45.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:16:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-28T11:16:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-28T11:16:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00025215 s, 2.0 MB/s 2023-11-28T11:16:45.479 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:16:45.526 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-28 11:16:04.424395975 +0000 2023-11-28T11:16:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-28 11:12:46.013290356 +0000 2023-11-28T11:16:45.575 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-28 11:12:46.013290356 +0000 2023-11-28T11:16:45.575 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-11-28 11:12:45.991290734 +0000 2023-11-28T11:16:45.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:16:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-28T11:16:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-28T11:16:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000170756 s, 3.0 MB/s 2023-11-28T11:16:45.683 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:16:45.730 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-11-28T11:16:45.775 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-28 11:16:04.425395957 +0000 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-28 11:12:46.829276339 +0000 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-28 11:12:46.829276339 +0000 2023-11-28T11:16:45.776 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-11-28 11:12:46.806276734 +0000 2023-11-28T11:16:45.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:16:45.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-28T11:16:45.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-28T11:16:45.843 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000191763 s, 2.7 MB/s 2023-11-28T11:16:45.846 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:16:45.893 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:16:45.938 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-28 11:16:04.425395957 +0000 2023-11-28T11:16:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-28 11:12:47.695261463 +0000 2023-11-28T11:16:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-28 11:12:47.695261463 +0000 2023-11-28T11:16:45.939 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-11-28 11:12:47.671261875 +0000 2023-11-28T11:16:45.939 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:16:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-28T11:16:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-28T11:16:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000174554 s, 2.9 MB/s 2023-11-28T11:16:46.048 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:16:46.093 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:46.093 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:16:46.094 INFO:tasks.ceph:Generating config... 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:16:46.095 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:16:46.095 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T11:16:46.095 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:16:46.096 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:16:46.097 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:16:46.097 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:16:46.098 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:16:46.098 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:16:46.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:16:46.242 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:16:46.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:16:46.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:16:46.424 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.53'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2023-11-28T11:16:46.424 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.53,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:16:46.425 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:46.425 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:16:46.470 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.53 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:46.603 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:46.603 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: generated fsid 38901cf1-eb03-4cf6-8c7d-44ad237eb202 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = pacific 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 0 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 38901cf1-eb03-4cf6-8c7d-44ad237eb202 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:last_changed 2023-11-28T11:16:46.603522+0000 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:created 2023-11-28T11:16:46.603522+0000 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:min_mon_release 16 (pacific) 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:election_strategy: 1 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2023-11-28T11:16:46.604 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:16:46.607 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:16:46.650 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 38901cf1-eb03-4cf6-8c7d-44ad237eb202... 2023-11-28T11:16:46.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:16:46.695 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:16:46.816 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-28T11:16:46.817 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.818 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:16:46.819 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.53,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 38901cf1-eb03-4cf6-8c7d-44ad237eb202 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-28T11:16:46.820 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-28T11:16:46.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.822 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:16:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-28T11:16:46.824 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.825 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-28T11:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-28T11:16:46.827 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-28T11:16:46.828 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2023-11-28T11:16:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:16:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2023-11-28T11:16:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2023-11-28T11:16:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 38901cf1-eb03-4cf6-8c7d-44ad237eb202 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2023-11-28T11:16:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2023-11-28T11:16:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bdev async discard = True 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bdev enable discard = True 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore allocator = bitmap 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:16:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.836 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:16:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2023-11-28T11:16:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2023-11-28T11:16:46.839 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2023-11-28T11:16:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2023-11-28T11:16:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2023-11-28T11:16:46.853 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:16:46.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:16:46.974 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:16:46.974 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:46.974 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:16:47.018 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:47.018 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:16:47.103 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:16:47.103 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:47.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:16:47.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:16:47.242 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:16:47.242 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:47.327 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:16:47.327 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:47.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:16:47.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:16:47.427 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:16:47.428 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:47.512 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:16:47.512 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:16:47.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:16:47.645 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:16:47.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:16:47.790 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:16:47.797 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:16:47.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:16:47.927 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:16:47.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:16:48.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:16:48.140 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:16:48.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:16:48.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:16:48.347 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:16:48.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:16:48.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:16:48.559 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:16:48.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:16:48.633 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:16:48.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:16:48.738 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:16:48.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:16:48.874 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:16:48.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:16:49.006 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:16:49.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:16:49.139 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:16:49.172 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:16:49.173 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:16:49.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:16:49.246 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:49.246 INFO:tasks.ceph:role: osd.0 2023-11-28T11:16:49.247 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:16:49.247 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:16:49.317 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:16:49.318 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:16:49.318 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:16:49.318 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:16:49.340 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-11-28T11:16:49.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-11-28T11:16:49.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:16:49.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:16:49.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:16:49.611 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:49.611 INFO:tasks.ceph:role: osd.1 2023-11-28T11:16:49.612 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:16:49.612 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-11-28T11:16:49.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-11-28T11:16:49.710 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:16:49.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:16:49.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:16:49.969 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:49.970 INFO:tasks.ceph:role: osd.2 2023-11-28T11:16:49.970 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:16:49.970 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:16:50.041 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:16:50.042 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:16:50.065 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-11-28T11:16:50.069 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-11-28T11:16:50.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:16:50.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:16:50.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:16:50.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:16:50.327 INFO:tasks.ceph:role: osd.3 2023-11-28T11:16:50.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:16:50.327 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:16:50.397 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:16:50.397 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:16:50.398 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:16:50.421 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-11-28T11:16:50.425 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-11-28T11:16:50.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:16:50.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:16:50.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:51.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.527+0000 7f1e65939640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:16:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.527+0000 7f1e65939640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:16:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.527+0000 7f1e65939640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:16:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.528+0000 7f1e65939640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:16:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.528+0000 7f1e65939640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:16:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:51.530+0000 7f1e65939640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:16:54.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:16:54.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.650+0000 7ffabfc4f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:16:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.650+0000 7ffabfc4f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:16:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.650+0000 7ffabfc4f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:16:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.650+0000 7ffabfc4f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:16:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.650+0000 7ffabfc4f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:16:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:54.652+0000 7ffabfc4f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:16:57.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:16:57.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:16:57.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.775+0000 7fdb4adb0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:16:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.776+0000 7fdb4adb0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:16:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.776+0000 7fdb4adb0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:16:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.776+0000 7fdb4adb0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:16:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.776+0000 7fdb4adb0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:16:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:16:57.777+0000 7fdb4adb0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:17:00.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:17:00.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:00.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.884+0000 7f8ff0b95640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:17:00.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.884+0000 7f8ff0b95640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:17:00.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.884+0000 7f8ff0b95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:17:00.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.884+0000 7f8ff0b95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:17:00.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.884+0000 7f8ff0b95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:17:00.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:00.886+0000 7f8ff0b95640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:17:03.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:17:03.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:17:03.985 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:17:03.985 INFO:tasks.ceph:role: osd.4 2023-11-28T11:17:03.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:17:03.986 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:17:04.055 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:17:04.056 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:17:04.079 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-28T11:17:04.083 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-28T11:17:04.083 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:17:04.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:17:04.268 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:17:04.336 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:17:04.337 INFO:tasks.ceph:role: osd.5 2023-11-28T11:17:04.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:17:04.337 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:17:04.404 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:17:04.405 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:17:04.427 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-28T11:17:04.431 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-28T11:17:04.431 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:17:04.539 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:17:04.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:17:04.885 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:17:04.885 INFO:tasks.ceph:role: osd.6 2023-11-28T11:17:04.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:17:04.886 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:17:04.953 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:17:04.953 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:17:04.953 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:17:04.953 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:17:04.954 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:17:04.977 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-28T11:17:04.980 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-28T11:17:04.980 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:17:05.085 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:17:05.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:17:05.230 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:17:05.230 INFO:tasks.ceph:role: osd.7 2023-11-28T11:17:05.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:17:05.230 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:17:05.298 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:17:05.298 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:17:05.298 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:17:05.299 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:17:05.321 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-28T11:17:05.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-28T11:17:05.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:17:05.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:17:05.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.625+0000 7f103ee5a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:17:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.625+0000 7f103ee5a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:17:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.625+0000 7f103ee5a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:17:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.625+0000 7f103ee5a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:17:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.625+0000 7f103ee5a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:17:05.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:05.627+0000 7f103ee5a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:17:08.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:17:08.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:08.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.692+0000 7f55a8f29640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:17:08.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.692+0000 7f55a8f29640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:17:08.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.692+0000 7f55a8f29640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:17:08.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.692+0000 7f55a8f29640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:17:08.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.692+0000 7f55a8f29640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:17:08.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:08.694+0000 7f55a8f29640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:17:11.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:17:11.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.759+0000 7f85b215f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:17:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.759+0000 7f85b215f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:17:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.759+0000 7f85b215f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:17:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.759+0000 7f85b215f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:17:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.759+0000 7f85b215f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:17:11.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:11.761+0000 7f85b215f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:17:14.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:17:14.111 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:17:14.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:17:14.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.813+0000 7f93c9a64640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:17:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.813+0000 7f93c9a64640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:17:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.813+0000 7f93c9a64640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:17:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.813+0000 7f93c9a64640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:17:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.813+0000 7f93c9a64640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:17:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:14.815+0000 7f93c9a64640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:17:17.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:17:17.767 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:17:17.767 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:17.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:17:17.841 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:17.841 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:17:17.952 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:17.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:17:18.063 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:18.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:17:18.175 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:18.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:17:18.289 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:18.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:17:18.402 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:18.402 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:17:18.513 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:18.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:17:18.583 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:18.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:17:18.691 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:18.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:17:18.799 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:18.799 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:17:18.907 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:18.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:17:19.016 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:19.017 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:17:19.125 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:19.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:17:19.233 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:19.233 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:17:19.276 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:19.277 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:17:19.360 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:19.360 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:17:19.403 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:19.403 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:17:19.485 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:17:19.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:17:19.490 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:17:19.552 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAfzGVl91F6JhAAGc79E5/TfWduB198lkdQqw== 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAfzGVlNkJINxAAwB6S1pnMrZk3sG6i+8PR9Q== 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAgzGVlH5ZYCBAAdBiB6rthc32/VYiX/UUEpg== 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAjzGVlnYF9HxAAx5MdeV6qs7Ce0YL/BzjUuQ== 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAmzGVlheXPJhAAqVOmYx93P2jBYNhzlsg4EQ== 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2023-11-28T11:17:19.553 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQApzGVlRXhLLhAAcCx5wY3VywAx4UWnz15sOg== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAszGVlxc3CNBAAo8dwkn6w3PZYcNFkTdxZEA== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAfzGVllCcfLxAAjmT46vM+3CEOyE6aCbidrQ== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAgzGVlHGGyFBAAkLNG+TdDzS8Rd+Q2LV6BYw== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAgzGVlUdNXIRAAete67odjh+SjYr5KUAmDvw== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAxzGVlKUZYJRAALIbrf8b2mXrVTt8ebMFSnA== 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2023-11-28T11:17:19.554 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA0zGVl2RxXKRAAyHpuvnKEAS2o9BNE1LdRGw== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA3zGVlFw1WLRAAf+SrDy5UyYmdT3NfXFYA2A== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA6zGVlCqCMMBAAq6pur3Ycnp4CiTD40y4upw== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAgzGVlvIIFLBAAxrqCs6enHKg93ih+QHQcdw== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[client.2] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAgzGVlrXsWNBAA+cEN015xjMYKa6JKavZobQ== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAhzGVlkkNgABAAD7UONIytbl4yWdjvGODhXw== 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:[client.3] 2023-11-28T11:17:19.555 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAhzGVl2rlMCBAA0jfD14UyD2AjgyjtkShrTw== 2023-11-28T11:17:19.590 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAfzGVl91F6JhAAGc79E5/TfWduB198lkdQqw== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAfzGVlNkJINxAAwB6S1pnMrZk3sG6i+8PR9Q== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgzGVlH5ZYCBAAdBiB6rthc32/VYiX/UUEpg== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAjzGVlnYF9HxAAx5MdeV6qs7Ce0YL/BzjUuQ== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAmzGVlheXPJhAAqVOmYx93P2jBYNhzlsg4EQ== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQApzGVlRXhLLhAAcCx5wY3VywAx4UWnz15sOg== 2023-11-28T11:17:19.591 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAszGVlxc3CNBAAo8dwkn6w3PZYcNFkTdxZEA== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAfzGVllCcfLxAAjmT46vM+3CEOyE6aCbidrQ== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgzGVlHGGyFBAAkLNG+TdDzS8Rd+Q2LV6BYw== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgzGVlUdNXIRAAete67odjh+SjYr5KUAmDvw== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAxzGVlKUZYJRAALIbrf8b2mXrVTt8ebMFSnA== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA0zGVl2RxXKRAAyHpuvnKEAS2o9BNE1LdRGw== 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2023-11-28T11:17:19.592 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA3zGVlFw1WLRAAf+SrDy5UyYmdT3NfXFYA2A== 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA6zGVlCqCMMBAAq6pur3Ycnp4CiTD40y4upw== 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgzGVlvIIFLBAAxrqCs6enHKg93ih+QHQcdw== 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAgzGVlrXsWNBAA+cEN015xjMYKa6JKavZobQ== 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAhzGVlkkNgABAAD7UONIytbl4yWdjvGODhXw== 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2023-11-28T11:17:19.593 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAhzGVl2rlMCBAA0jfD14UyD2AjgyjtkShrTw== 2023-11-28T11:17:19.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:17:19.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:17:19.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:19.748 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:19.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:19.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:19.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:19.984 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:17:20.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:17:20.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:20.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:20.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:20.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:17:20.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:20.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:21.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:21.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:21.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:21.139 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:17:21.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:17:21.715 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:17:21.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:17:21.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:17:21.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:17:21.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:17:22.057 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:17:22.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:17:22.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:17:22.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:17:22.442 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:17:22.510 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:22.515 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:17:22.558 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:17:22.558 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:17:22.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:17:22.599 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:17:22.599 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:17:22.599 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:17:22.604 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:17:22.604 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:17:22.604 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:17:22.608 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:17:22.608 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:17:22.608 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:17:22.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:17:22.613 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:17:22.613 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:17:22.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:17:22.617 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:17:22.617 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:17:22.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:17:22.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.829+0000 7f90b79ff700 1 Processor -- start 2023-11-28T11:17:22.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.830+0000 7f90b79ff700 1 -- start start 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b79ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b79ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00d1950 con 0x7f90b015c9b0 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b79ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00ce140 con 0x7f90b0161340 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b79ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00b5800 con 0x7f90b00cd7c0 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b579b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f90b015c9b0 0x7f90b00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54642/0 (socket says 172.21.15.53:54642) 2023-11-28T11:17:22.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 learned_addr learned my addr 172.21.15.53:0/2208716047 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:22.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.831+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:22.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.832+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:22.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.832+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:22.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.832+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:22.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:22.832+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.033+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.033+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.033+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:23.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.033+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:23.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.033+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T11:17:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.434+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.434+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.434+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.434+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:23.434+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T11:17:24.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:24.235+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:24.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:24.235+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:24.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:24.235+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:24.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:24.235+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:24.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:24.235+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 msgr2=0x7f90b00cdc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00cd7c0 0x7f90b00cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 >> v1:172.21.15.157:6789/0 conn(0x7f90b0161340 legacy=0x7f90b00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 >> v1:172.21.15.53:6789/0 conn(0x7f90b015c9b0 legacy=0x7f90b00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00b5800 con 0x7f9094003780 2023-11-28T11:17:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9094008db0 con 0x7f9094006240 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.831+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:25.832+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:26.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:26.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:26.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.033+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T11:17:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:26.434+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T11:17:27.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:27.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:27.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b579b700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b579b700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:27.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:27.236+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T11:17:27.979 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:27.978+0000 7f09d56c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:17:27.985 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:27.984+0000 7fe4bbac8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:17:28.063 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:28.062+0000 7f09d56c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:17:28.071 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:28.070+0000 7fe4bbac8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:17:28.157 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:28.156+0000 7f09d56c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:17:28.169 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:28.168+0000 7fe4bbac8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:17:28.256 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:28.255+0000 7f09d56c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:17:28.271 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:28.270+0000 7fe4bbac8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:17:28.437 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:28.436+0000 7f09d56c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:17:28.457 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:28.456+0000 7fe4bbac8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:17:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 msgr2=0x7f90940086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006240 0x7f90940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 msgr2=0x7f9094005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003780 0x7f9094005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2208716047 reap_dead start 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003be0 0x7f90940066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006c20 0x7f90940070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00d24f0 con 0x7f90b015c9b0 2023-11-28T11:17:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00b5800 con 0x7f9094003be0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.832+0000 7f90a6ffd700 1 -- 172.21.15.53:0/2208716047 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9094008db0 con 0x7f9094006c20 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.53:3300/0 conn(0x7f9094003be0 0x7f90940066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006c20 0x7f90940070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006c20 msgr2=0x7f90940070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006c20 0x7f90940070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 >> v1:172.21.15.53:6789/0 conn(0x7f90b015c9b0 legacy=0x7f909400b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2208716047 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094008760 con 0x7f9094003be0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094003be0 0x7f90940066e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9094006060 tx=0x7f90a0005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef7881a1e3555c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90a7fff700 1 -- 172.21.15.53:0/2208716047 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0016450 con 0x7f9094003be0 2023-11-28T11:17:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.833+0000 7f90a7fff700 1 -- 172.21.15.53:0/2208716047 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a00165d0 con 0x7f9094003be0 2023-11-28T11:17:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90a7fff700 1 -- 172.21.15.53:0/2208716047 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0016980 con 0x7f9094003be0 2023-11-28T11:17:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 -- 172.21.15.53:0/2208716047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094003be0 msgr2=0x7f90940066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094003be0 0x7f90940066e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9094006060 tx=0x7f90a0005610 comp rx=0 tx=0).stop 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 -- 172.21.15.53:0/2208716047 shutdown_connections 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2208716047 >> v2:172.21.15.157:3300/0 conn(0x7f9094006c20 0x7f90940070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2208716047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094003be0 0x7f90940066e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 -- 172.21.15.53:0/2208716047 >> 172.21.15.53:0/2208716047 conn(0x7f90b0072600 msgr2=0x7f90b00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 -- 172.21.15.53:0/2208716047 shutdown_connections 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.834+0000 7f90b79ff700 1 -- 172.21.15.53:0/2208716047 wait complete. 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.836+0000 7f90b79ff700 1 Processor -- start 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.836+0000 7f90b79ff700 1 -- start start 2023-11-28T11:17:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 0x7f90b015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cbf30 con 0x7f9094006c20 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cc210 con 0x7f90b015c9b0 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.837+0000 7f90b79ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cc4f0 con 0x7f9094003be0 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b4f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b4f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45956/0 (socket says 172.21.15.53:45956) 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b579b700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b4f9a700 1 -- 172.21.15.53:0/2140386910 learned_addr learned my addr 172.21.15.53:0/2140386910 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 0x7f90b015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b579b700 1 -- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 msgr2=0x7f90b01e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b579b700 1 -- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 msgr2=0x7f90b01e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b579b700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b579b700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 msgr2=0x7f90b01e7830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2140386910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 msgr2=0x7f90b015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 -- 172.21.15.53:0/2140386910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a000f040 con 0x7f90b015c9b0 2023-11-28T11:17:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b4f9a700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.838+0000 7f90b5f9c700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 0x7f90b015eb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90b00d2960 tx=0x7f90a8004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80178e0 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a800f040 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8017c60 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b00d1300 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00b5520 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f90a8020750 con 0x7f90b015c9b0 2023-11-28T11:17:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.839+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f90a80156c0 con 0x7f90b015c9b0 2023-11-28T11:17:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.840+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b00d15e0 con 0x7f90b015c9b0 2023-11-28T11:17:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.843+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f90a801b070 con 0x7f90b015c9b0 2023-11-28T11:17:28.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.924+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f90b00cc630 con 0x7f90b015c9b0 2023-11-28T11:17:28.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.928+0000 7f90a5ffb700 1 -- 172.21.15.53:0/2140386910 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f90b00cc630 con 0x7f90b015c9b0 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 msgr2=0x7f90b015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 0x7f90b015eb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90b00d2960 tx=0x7f90a8004530 comp rx=0 tx=0).stop 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 shutdown_connections 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094003be0 0x7f90b01e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b015c9b0 0x7f90b015eb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 --2- 172.21.15.53:0/2140386910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9094006c20 0x7f90b015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:28.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 >> 172.21.15.53:0/2140386910 conn(0x7f90b0072600 msgr2=0x7f90b00b8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:28.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 shutdown_connections 2023-11-28T11:17:28.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:28.929+0000 7f90b79ff700 1 -- 172.21.15.53:0/2140386910 wait complete. 2023-11-28T11:17:28.931 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2023-11-28T11:17:28.948 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:17:28.948 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:17:28.949 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:28.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:17:29.020 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:29.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:17:29.131 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:29.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:17:29.247 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:17:29.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:17:29.296 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:29.295+0000 7f09d56c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:17:29.327 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:29.326+0000 7fe4bbac8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:17:29.363 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:29.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:17:29.395 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:29.394+0000 7f09d56c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:17:29.429 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:29.428+0000 7fe4bbac8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:17:29.432 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:29.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:17:29.507 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:29.506+0000 7f09d56c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:17:29.535 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:29.534+0000 7fe4bbac8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:17:29.543 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:29.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:17:29.652 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:17:29.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:17:29.698 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:29.697+0000 7f09d56c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:17:29.706 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:29.706+0000 7fe4bbac8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:17:29.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 03c80b06-e2ed-4b09-ae77-76cbf0f42bcd 0 2023-11-28T11:17:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.993+0000 7f0b2bdff700 1 Processor -- start 2023-11-28T11:17:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.993+0000 7f0b2bdff700 1 -- start start 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2bdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b240d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2bdff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b240d1950 con 0x7f0b2415c9b0 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2bdff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b240b5520 con 0x7f0b240d2320 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2bdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b5800 con 0x7f0b24161340 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2939a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b240d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2a39c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0b240d2320 0x7f0b240cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58486/0 (socket says 172.21.15.157:58486) 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.994+0000 7f0b2a39c700 1 -- 172.21.15.157:0/116994221 learned_addr learned my addr 172.21.15.157:0/116994221 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b2939a700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 msgr2=0x7f0b240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b2939a700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 msgr2=0x7f0b240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b2939a700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b240d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b2939a700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b240d1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128791858 0 0) 0x7f0b240b5520 con 0x7f0b240d2320 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b100035b0 con 0x7f0b240d2320 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649058108 0 0) 0x7f0b240d1950 con 0x7f0b2415c9b0 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b240b5520 con 0x7f0b2415c9b0 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643138865 0 0) 0x7f0b100035b0 con 0x7f0b240d2320 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 msgr2=0x7f0b240d1de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b240d1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 >> v1:172.21.15.53:6789/0 conn(0x7f0b2415c9b0 legacy=0x7f0b240b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:29.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240d1300 con 0x7f0b240d2320 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f0b1c0019f0 con 0x7f0b240d2320 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 >> v1:172.21.15.157:6789/0 conn(0x7f0b240d2320 legacy=0x7f0b240cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10004c20 0x7f0b10004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.995+0000 7f0b28b99700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b100037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10004c20 0x7f0b10004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b28b99700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10003d30 0x7f0b100041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20001230 con 0x7f0b10003d30 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b5800 con 0x7f0b10004c20 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1000bbd0 con 0x7f0b24161340 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b2939a700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b100037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b2a39c700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10003d30 0x7f0b100041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 msgr2=0x7f0b100037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b24161340 0x7f0b100037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10003d30 msgr2=0x7f0b100041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10003d30 0x7f0b100041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 -- 172.21.15.157:0/116994221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100035b0 con 0x7f0b10004c20 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b2a39c700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10003d30 0x7f0b100041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.996+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10004c20 0x7f0b10004ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f50 tx=0x7f0b10005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1ffbdd1d23f7c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200052d0 con 0x7f0b10004c20 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20005450 con 0x7f0b10004c20 2023-11-28T11:17:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b28b99700 1 -- 172.21.15.157:0/116994221 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000cc10 con 0x7f0b10004c20 2023-11-28T11:17:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b2bdff700 1 -- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10004c20 msgr2=0x7f0b10004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b2bdff700 1 --2- 172.21.15.157:0/116994221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10004c20 0x7f0b10004ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f50 tx=0x7f0b10005880 comp rx=0 tx=0).stop 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b2a39c700 1 -- 172.21.15.157:0/116994221 reap_dead start 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b2bdff700 1 -- 172.21.15.157:0/116994221 shutdown_connections 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.997+0000 7f0b2bdff700 1 -- 172.21.15.157:0/116994221 >> 172.21.15.157:0/116994221 conn(0x7f0b24072600 msgr2=0x7f0b240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.998+0000 7f0b2bdff700 1 -- 172.21.15.157:0/116994221 shutdown_connections 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:29.998+0000 7f0b2bdff700 1 -- 172.21.15.157:0/116994221 wait complete. 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.000+0000 7f0b2bdff700 1 Processor -- start 2023-11-28T11:17:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.001+0000 7f0b2bdff700 1 -- start start 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 0x7f0b240c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b240c8cb0 0x7f0b241514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240c71e0 con 0x7f0b10004c20 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241576a0 con 0x7f0b24156cf0 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2bdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240c9ef0 con 0x7f0b240c8cb0 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2a39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2939a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b240c8cb0 0x7f0b241514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2a39c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38646/0 (socket says 172.21.15.157:38646) 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.002+0000 7f0b2a39c700 1 -- 172.21.15.157:0/3264239555 learned_addr learned my addr 172.21.15.157:0/3264239555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 0x7f0b240c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 -- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b240c8cb0 msgr2=0x7f0b241514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b240c8cb0 0x7f0b241514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 -- 172.21.15.157:0/3264239555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 msgr2=0x7f0b240c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 0x7f0b240c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 -- 172.21.15.157:0/3264239555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2415cd80 con 0x7f0b24156cf0 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b29b9b700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 0x7f0b240c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2a39c700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0b240685c0 tx=0x7f0b1c00df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00bb70 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00bcf0 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b240d15e0 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.003+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c007270 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.004+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2415cf10 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.004+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0b1c01b030 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.004+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0b1c0137b0 con 0x7f0b24156cf0 2023-11-28T11:17:30.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.005+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24153370 con 0x7f0b24156cf0 2023-11-28T11:17:30.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.007+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0b1c019070 con 0x7f0b24156cf0 2023-11-28T11:17:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.095+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "03c80b06-e2ed-4b09-ae77-76cbf0f42bcd", "id": 0} v 0) v1 -- 0x7f0b240745c0 con 0x7f0b24156cf0 2023-11-28T11:17:30.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.099+0000 7f0b1a7fc700 1 -- 172.21.15.157:0/3264239555 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03c80b06-e2ed-4b09-ae77-76cbf0f42bcd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b1c0073d0 con 0x7f0b24156cf0 2023-11-28T11:17:30.100 INFO:teuthology.orchestra.run.smithi157.stdout:0 2023-11-28T11:17:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.100+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 msgr2=0x7f0b24157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.100+0000 7f0b2bdff700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0b240685c0 tx=0x7f0b1c00df90 comp rx=0 tx=0).stop 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 shutdown_connections 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b24156cf0 0x7f0b24157160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b10004c20 0x7f0b240c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 --2- 172.21.15.157:0/3264239555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b240c8cb0 0x7f0b241514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 >> 172.21.15.157:0/3264239555 conn(0x7f0b24072600 msgr2=0x7f0b2406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 shutdown_connections 2023-11-28T11:17:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.101+0000 7f0b2bdff700 1 -- 172.21.15.157:0/3264239555 wait complete. 2023-11-28T11:17:30.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 67e5f4d1-25d5-4e62-a73f-85361c4c2a64 1 2023-11-28T11:17:30.208 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:30.207+0000 7fe4bbac8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:17:30.221 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:30.220+0000 7f09d56c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:17:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.316+0000 7f9cbc9ff700 1 Processor -- start 2023-11-28T11:17:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.317+0000 7f9cbc9ff700 1 -- start start 2023-11-28T11:17:30.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cbc9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9cb8161340 0x7f9cb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cbc9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9cb80d2320 0x7f9cb80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cbc9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb80d1950 con 0x7f9cb815c9b0 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cbc9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb80b5800 con 0x7f9cb8161340 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cbc9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb80b5300 con 0x7f9cb80d2320 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9cb80d2320 0x7f9cb80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9cb80d2320 0x7f9cb80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38650/0 (socket says 172.21.15.157:38650) 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 -- 172.21.15.157:0/4196950828 learned_addr learned my addr 172.21.15.157:0/4196950828 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9caffff700 1 --2- 172.21.15.157:0/4196950828 >> v2:172.21.15.53:3300/0 conn(0x7f9cb8161340 0x7f9cb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 -- 172.21.15.157:0/4196950828 >> v2:172.21.15.53:3300/0 conn(0x7f9cb8161340 msgr2=0x7f9cb80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 --2- 172.21.15.157:0/4196950828 >> v2:172.21.15.53:3300/0 conn(0x7f9cb8161340 0x7f9cb80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.318+0000 7f9cb6d9d700 1 -- 172.21.15.157:0/4196950828 >> v1:172.21.15.53:6789/0 conn(0x7f9cb815c9b0 legacy=0x7f9cb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.319+0000 7f9cb6d9d700 1 -- 172.21.15.157:0/4196950828 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80b5520 con 0x7f9cb80d2320 2023-11-28T11:17:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.319+0000 7f9cb6d9d700 1 --2- 172.21.15.157:0/4196950828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80cd7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806c170 tx=0x7f9cb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a8ef460d465000 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.319+0000 7f9cb5d9b700 1 -- 172.21.15.157:0/4196950828 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000e070 con 0x7f9cb80d2320 2023-11-28T11:17:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.319+0000 7f9cb5d9b700 1 -- 172.21.15.157:0/4196950828 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb000b910 con 0x7f9cb80d2320 2023-11-28T11:17:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.319+0000 7f9cb5d9b700 1 -- 172.21.15.157:0/4196950828 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000bcc0 con 0x7f9cb80d2320 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/4196950828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 msgr2=0x7f9cb80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/4196950828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80cd7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806c170 tx=0x7f9cb000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/4196950828 shutdown_connections 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/4196950828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80cd7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/4196950828 >> v2:172.21.15.53:3300/0 conn(0x7f9cb8161340 0x7f9cb80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/4196950828 >> 172.21.15.157:0/4196950828 conn(0x7f9cb8072600 msgr2=0x7f9cb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/4196950828 shutdown_connections 2023-11-28T11:17:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.320+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/4196950828 wait complete. 2023-11-28T11:17:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.323+0000 7f9cbc9ff700 1 Processor -- start 2023-11-28T11:17:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.323+0000 7f9cbc9ff700 1 -- start start 2023-11-28T11:17:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb8161340 0x7f9cb814a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81e73d0 con 0x7f9cb8161340 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb814aac0 con 0x7f9cb80d2320 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.324+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81e7670 con 0x7f9cb815c9b0 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9caffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38656/0 (socket says 172.21.15.157:38656) 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 -- 172.21.15.157:0/2826073318 learned_addr learned my addr 172.21.15.157:0/2826073318 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb6d9d700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb8161340 0x7f9cb814a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9caffff700 1 -- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 msgr2=0x7f9cb81e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9caffff700 1 -- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 msgr2=0x7f9cb81e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9caffff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9caffff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 -- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 msgr2=0x7f9cb81e6e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 -- 172.21.15.157:0/2826073318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb8161340 msgr2=0x7f9cb814a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb8161340 0x7f9cb814a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 -- 172.21.15.157:0/2826073318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0009b30 con 0x7f9cb80d2320 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.325+0000 7f9cb659c700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c450 tx=0x7f9ca800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80052d0 con 0x7f9cb80d2320 2023-11-28T11:17:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca8005430 con 0x7f9cb80d2320 2023-11-28T11:17:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80038c0 con 0x7f9cb80d2320 2023-11-28T11:17:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb80d15e0 con 0x7f9cb80d2320 2023-11-28T11:17:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb80d1300 con 0x7f9cb80d2320 2023-11-28T11:17:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9ca8019030 con 0x7f9cb80d2320 2023-11-28T11:17:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.326+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f9ca80118f0 con 0x7f9cb80d2320 2023-11-28T11:17:30.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.327+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb815cd80 con 0x7f9cb80d2320 2023-11-28T11:17:30.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.330+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9ca8017070 con 0x7f9cb80d2320 2023-11-28T11:17:30.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.433+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "67e5f4d1-25d5-4e62-a73f-85361c4c2a64", "id": 1} v 0) v1 -- 0x7f9cb80745c0 con 0x7f9cb80d2320 2023-11-28T11:17:30.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.437+0000 7f9caeffd700 1 -- 172.21.15.157:0/2826073318 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "67e5f4d1-25d5-4e62-a73f-85361c4c2a64", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ca8011ab0 con 0x7f9cb80d2320 2023-11-28T11:17:30.438 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 msgr2=0x7f9cb80c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c450 tx=0x7f9ca800c760 comp rx=0 tx=0).stop 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 shutdown_connections 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb8161340 0x7f9cb814a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb80d2320 0x7f9cb80c9070 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 --2- 172.21.15.157:0/2826073318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb815c9b0 0x7f9cb81e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 >> 172.21.15.157:0/2826073318 conn(0x7f9cb8072600 msgr2=0x7f9cb80b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 shutdown_connections 2023-11-28T11:17:30.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.438+0000 7f9cbc9ff700 1 -- 172.21.15.157:0/2826073318 wait complete. 2023-11-28T11:17:30.456 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new c7f8e310-8aef-4cd5-9456-a50608f17a2f 2 2023-11-28T11:17:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.650+0000 7f9bcbfff700 1 Processor -- start 2023-11-28T11:17:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.650+0000 7f9bcbfff700 1 -- start start 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.651+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.651+0000 7f9bcbfff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40d1950 con 0x7f9bc4161340 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.651+0000 7f9bcbfff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40ce140 con 0x7f9bc415c9b0 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.651+0000 7f9bcbfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40b5800 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc9d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9bc415c9b0 0x7f9bc40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58492/0 (socket says 172.21.15.157:58492) 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/912786940 learned_addr learned my addr 172.21.15.157:0/912786940 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-28T11:17:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bca59c700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bca59c700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bca59c700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bca59c700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bca59c700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673363474 0 0) 0x7f9bc40ce140 con 0x7f9bc415c9b0 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb00035b0 con 0x7f9bc415c9b0 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965980721 0 0) 0x7f9bc40d1950 con 0x7f9bc4161340 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc40ce140 con 0x7f9bc4161340 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254858064 0 0) 0x7f9bb00035b0 con 0x7f9bc415c9b0 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cdc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 >> v1:172.21.15.53:6789/0 conn(0x7f9bc4161340 legacy=0x7f9bc40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40d24f0 con 0x7f9bc415c9b0 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f9bc0001670 con 0x7f9bc415c9b0 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 >> v1:172.21.15.157:6789/0 conn(0x7f9bc415c9b0 legacy=0x7f9bc40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bb0004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.652+0000 7f9bc8d99700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bb0005150 0x7f9bb00037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc8d99700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bb0003d30 0x7f9bb00041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40b5800 con 0x7f9bb0005150 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb000bbd0 con 0x7f9bb0003d30 2023-11-28T11:17:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb000bd90 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bb0004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc959a700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bb0005150 0x7f9bb00037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bca59c700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bb0003d30 0x7f9bb00041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bb0003d30 msgr2=0x7f9bb00041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bb0003d30 0x7f9bb00041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bb0005150 msgr2=0x7f9bb00037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bb0005150 0x7f9bb00037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/912786940 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00035b0 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.653+0000 7f9bc959a700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bb0005150 0x7f9bb00037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bb0004bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c210 tx=0x7f9bb0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f436771ef6b71899 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000cd50 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0005960 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bc8d99700 1 -- 172.21.15.157:0/912786940 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0005d10 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bcbfff700 1 -- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bb0004bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bcbfff700 1 --2- 172.21.15.157:0/912786940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bb0004bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c210 tx=0x7f9bb0005880 comp rx=0 tx=0).stop 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.654+0000 7f9bca59c700 1 -- 172.21.15.157:0/912786940 reap_dead start 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.655+0000 7f9bcbfff700 1 -- 172.21.15.157:0/912786940 shutdown_connections 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.655+0000 7f9bcbfff700 1 -- 172.21.15.157:0/912786940 >> 172.21.15.157:0/912786940 conn(0x7f9bc4072600 msgr2=0x7f9bc40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.655+0000 7f9bcbfff700 1 -- 172.21.15.157:0/912786940 shutdown_connections 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.655+0000 7f9bcbfff700 1 -- 172.21.15.157:0/912786940 wait complete. 2023-11-28T11:17:30.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.657+0000 7f9bcbfff700 1 Processor -- start 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.657+0000 7f9bcbfff700 1 -- start start 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bc41f5400 0x7f9bc415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 0x7f9bc41e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc415ed60 con 0x7f9bc41f5400 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4157f20 con 0x7f9bc415e8c0 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bcbfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40cc110 con 0x7f9bc40cd7c0 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:33898/0 (socket says 172.21.15.157:33898) 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/90269676 learned_addr learned my addr 172.21.15.157:0/90269676 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:30.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 -- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 0x7f9bc41e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc959a700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bc41f5400 0x7f9bc415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bc9d9b700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 -- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cac50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 -- 172.21.15.157:0/90269676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bc41f5400 msgr2=0x7f9bc415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bc41f5400 0x7f9bc415e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.658+0000 7f9bca59c700 1 -- 172.21.15.157:0/90269676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40d1300 con 0x7f9bc415e8c0 2023-11-28T11:17:30.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bca59c700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 0x7f9bc41e6dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bc408c4d0 tx=0x7f9bbc00f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc009ea0 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc003a20 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc40b5520 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc003dd0 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc40d15e0 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9bbc01b020 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.659+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f9bbc00fc80 con 0x7f9bc415e8c0 2023-11-28T11:17:30.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.660+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc40745c0 con 0x7f9bc415e8c0 2023-11-28T11:17:30.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.662+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9bb4001590 con 0x7f9bc415e8c0 2023-11-28T11:17:30.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.749+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c7f8e310-8aef-4cd5-9456-a50608f17a2f", "id": 2} v 0) v1 -- 0x7f9bc40d08a0 con 0x7f9bc415e8c0 2023-11-28T11:17:30.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.754+0000 7f9bba7fc700 1 -- 172.21.15.157:0/90269676 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7f8e310-8aef-4cd5-9456-a50608f17a2f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bbc004020 con 0x7f9bc415e8c0 2023-11-28T11:17:30.755 INFO:teuthology.orchestra.run.smithi157.stdout:2 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 msgr2=0x7f9bc41e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 0x7f9bc41e6dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bc408c4d0 tx=0x7f9bbc00f430 comp rx=0 tx=0).stop 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 shutdown_connections 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40cd7c0 0x7f9bc40cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc415e8c0 0x7f9bc41e6dc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 --2- 172.21.15.157:0/90269676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bc41f5400 0x7f9bc415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 >> 172.21.15.157:0/90269676 conn(0x7f9bc4072600 msgr2=0x7f9bc406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 shutdown_connections 2023-11-28T11:17:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.755+0000 7f9bcbfff700 1 -- 172.21.15.157:0/90269676 wait complete. 2023-11-28T11:17:30.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new d0389301-550a-4c46-9f78-a5ce6c079987 3 2023-11-28T11:17:30.942 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:30.941+0000 7fe4bbac8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:17:30.997 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:30.996+0000 7f09d56c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:17:30.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.996+0000 7f6291dff700 1 Processor -- start 2023-11-28T11:17:30.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.996+0000 7f6291dff700 1 -- start start 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.997+0000 7f6291dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.997+0000 7f6291dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f628c0b20b0 0x7f628c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.997+0000 7f6291dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0b0230 con 0x7f628c0b29b0 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6291dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c15cec0 con 0x7f628c0b20b0 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6291dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b4570 con 0x7f628c0d01b0 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f628c0b20b0 0x7f628c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f628c0b20b0 0x7f628c0d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38672/0 (socket says 172.21.15.157:38672) 2023-11-28T11:17:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:33900/0 (socket says 172.21.15.157:33900) 2023-11-28T11:17:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 -- 172.21.15.157:0/3579797438 learned_addr learned my addr 172.21.15.157:0/3579797438 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 -- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 msgr2=0x7f628c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 -- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 msgr2=0x7f628c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628b7fe700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 -- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 msgr2=0x7f628c0d0c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 -- 172.21.15.157:0/3579797438 >> v1:172.21.15.53:6789/0 conn(0x7f628c0b29b0 legacy=0x7f628c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f6283fff700 1 -- 172.21.15.157:0/3579797438 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0b4850 con 0x7f628c0b20b0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.998+0000 7f628affd700 1 -- 172.21.15.157:0/3579797438 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187695255 0 0) 0x7f628c0b0230 con 0x7f628c0b29b0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6283fff700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b20b0 0x7f628c0d1160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740098f0 tx=0x7f6274009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57f4e0adbbf9baa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f628affd700 1 -- 172.21.15.157:0/3579797438 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400b8a0 con 0x7f628c0b20b0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f628affd700 1 -- 172.21.15.157:0/3579797438 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400ba00 con 0x7f628c0b20b0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f628affd700 1 -- 172.21.15.157:0/3579797438 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400bd90 con 0x7f628c0b20b0 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6291dff700 1 -- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b20b0 msgr2=0x7f628c0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6291dff700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b20b0 0x7f628c0d1160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740098f0 tx=0x7f6274009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6291dff700 1 -- 172.21.15.157:0/3579797438 shutdown_connections 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6291dff700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0d01b0 0x7f628c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:30.999+0000 7f6291dff700 1 --2- 172.21.15.157:0/3579797438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b20b0 0x7f628c0d1160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.000+0000 7f6291dff700 1 -- 172.21.15.157:0/3579797438 >> 172.21.15.157:0/3579797438 conn(0x7f628c09c3f0 msgr2=0x7f628c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.000+0000 7f6291dff700 1 -- 172.21.15.157:0/3579797438 shutdown_connections 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.000+0000 7f6291dff700 1 -- 172.21.15.157:0/3579797438 wait complete. 2023-11-28T11:17:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.002+0000 7f6291dff700 1 Processor -- start 2023-11-28T11:17:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.002+0000 7f6291dff700 1 -- start start 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 0x7f628c0b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f628c0d01b0 0x7f628c1e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b7ed0 con 0x7f628c0d01b0 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b72c0 con 0x7f628c0b29b0 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.003+0000 7f6291dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b75a0 con 0x7f628c0b20b0 2023-11-28T11:17:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 0x7f628c0b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:33904/0 (socket says 172.21.15.157:33904) 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 -- 172.21.15.157:0/3099375812 learned_addr learned my addr 172.21.15.157:0/3099375812 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628bfff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f628c0d01b0 0x7f628c1e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 -- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 msgr2=0x7f628c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 -- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 msgr2=0x7f628c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f628b7fe700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 -- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 msgr2=0x7f628c1524e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 -- 172.21.15.157:0/3099375812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f628c0d01b0 msgr2=0x7f628c1e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f628c0d01b0 0x7f628c1e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 -- 172.21.15.157:0/3099375812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009550 con 0x7f628c0b29b0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.004+0000 7f6283fff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 0x7f628c0b6670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6274016000 tx=0x7f6274004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274004300 con 0x7f628c0b29b0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627401f070 con 0x7f628c0b29b0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62740036a0 con 0x7f628c0b29b0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0afec0 con 0x7f628c0b29b0 2023-11-28T11:17:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0afbe0 con 0x7f628c0b29b0 2023-11-28T11:17:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f627402a020 con 0x7f628c0b29b0 2023-11-28T11:17:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.005+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f627401ab10 con 0x7f628c0b29b0 2023-11-28T11:17:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.006+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0b2600 con 0x7f628c0b29b0 2023-11-28T11:17:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.010+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6274015020 con 0x7f628c0b29b0 2023-11-28T11:17:31.033 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:31.033+0000 7fe4bbac8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:17:31.092 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:31.091+0000 7f09d56c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:17:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.096+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d0389301-550a-4c46-9f78-a5ce6c079987", "id": 3} v 0) v1 -- 0x7f628c0b0200 con 0x7f628c0b29b0 2023-11-28T11:17:31.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.101+0000 7f6288ff9700 1 -- 172.21.15.157:0/3099375812 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0389301-550a-4c46-9f78-a5ce6c079987", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f627401a680 con 0x7f628c0b29b0 2023-11-28T11:17:31.102 INFO:teuthology.orchestra.run.smithi157.stdout:3 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 msgr2=0x7f628c0b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 0x7f628c0b6670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6274016000 tx=0x7f6274004750 comp rx=0 tx=0).stop 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 shutdown_connections 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f628c0b20b0 0x7f628c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f628c0d01b0 0x7f628c1e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 --2- 172.21.15.157:0/3099375812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f628c0b29b0 0x7f628c0b6670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.102+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 >> 172.21.15.157:0/3099375812 conn(0x7f628c09c3f0 msgr2=0x7f628c0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.103+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 shutdown_connections 2023-11-28T11:17:31.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.103+0000 7f6291dff700 1 -- 172.21.15.157:0/3099375812 wait complete. 2023-11-28T11:17:31.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 7d8fd8c6-573a-4c8f-b45a-9bd517392dc2 4 2023-11-28T11:17:31.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.310+0000 7f544ddff700 1 Processor -- start 2023-11-28T11:17:31.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.310+0000 7f544ddff700 1 -- start start 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.311+0000 7f544ddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.311+0000 7f544ddff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54480d2320 0x7f54480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f544ddff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54480d1950 con 0x7f544815c9b0 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f544ddff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480b5800 con 0x7f54480d2320 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f544ddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54480b5300 con 0x7f5448161340 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54480d2320 0x7f54480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54480d2320 0x7f54480cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38688/0 (socket says 172.21.15.157:38688) 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:33920/0 (socket says 172.21.15.157:33920) 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 -- 172.21.15.157:0/3034924408 learned_addr learned my addr 172.21.15.157:0/3034924408 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 -- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 -- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5446ffd700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 -- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480d1de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 -- 172.21.15.157:0/3034924408 >> v1:172.21.15.53:6789/0 conn(0x7f544815c9b0 legacy=0x7f54480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f5447fff700 1 -- 172.21.15.157:0/3034924408 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480b5520 con 0x7f54480d2320 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.312+0000 7f54467fc700 1 -- 172.21.15.157:0/3034924408 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723464504 0 0) 0x7f54480d1950 con 0x7f544815c9b0 2023-11-28T11:17:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f5447fff700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54480cd7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e80 tx=0x7f544000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c577cc8b626b4e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f54467fc700 1 -- 172.21.15.157:0/3034924408 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000e070 con 0x7f54480d2320 2023-11-28T11:17:31.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f54467fc700 1 -- 172.21.15.157:0/3034924408 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000b980 con 0x7f54480d2320 2023-11-28T11:17:31.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f54467fc700 1 -- 172.21.15.157:0/3034924408 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000bd30 con 0x7f54480d2320 2023-11-28T11:17:31.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f544ddff700 1 -- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 msgr2=0x7f54480cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.313+0000 7f544ddff700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54480cd7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e80 tx=0x7f544000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 -- 172.21.15.157:0/3034924408 shutdown_connections 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54480cd7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 --2- 172.21.15.157:0/3034924408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 -- 172.21.15.157:0/3034924408 >> 172.21.15.157:0/3034924408 conn(0x7f5448072600 msgr2=0x7f54480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 -- 172.21.15.157:0/3034924408 shutdown_connections 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.314+0000 7f544ddff700 1 -- 172.21.15.157:0/3034924408 wait complete. 2023-11-28T11:17:31.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.316+0000 7f544ddff700 1 Processor -- start 2023-11-28T11:17:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.316+0000 7f544ddff700 1 -- start start 2023-11-28T11:17:31.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f544815c9b0 0x7f5448154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5448155460 con 0x7f544815c9b0 2023-11-28T11:17:31.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5448155740 con 0x7f54480d2320 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.317+0000 7f544ddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5448155a20 con 0x7f5448161340 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38690/0 (socket says 172.21.15.157:38690) 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 -- 172.21.15.157:0/1782536561 learned_addr learned my addr 172.21.15.157:0/1782536561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5447fff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5446ffd700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f544815c9b0 0x7f5448154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5447fff700 1 -- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5447fff700 1 -- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5447fff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f5447fff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 -- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 msgr2=0x7f54480b80d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 -- 172.21.15.157:0/1782536561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f544815c9b0 msgr2=0x7f5448154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f544815c9b0 0x7f5448154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 -- 172.21.15.157:0/1782536561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5440009b30 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.318+0000 7f54477fe700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f543800c550 tx=0x7f543800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380054e0 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438003680 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438003a30 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54480d15e0 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54480d1300 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5438019030 con 0x7f54480d2320 2023-11-28T11:17:31.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.319+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f543801c440 con 0x7f54480d2320 2023-11-28T11:17:31.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.320+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544815cd80 con 0x7f54480d2320 2023-11-28T11:17:31.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.322+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5438017070 con 0x7f54480d2320 2023-11-28T11:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.412+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7d8fd8c6-573a-4c8f-b45a-9bd517392dc2", "id": 4} v 0) v1 -- 0x7f54480745c0 con 0x7f54480d2320 2023-11-28T11:17:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.416+0000 7f542ffff700 1 -- 172.21.15.157:0/1782536561 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d8fd8c6-573a-4c8f-b45a-9bd517392dc2", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5438003b90 con 0x7f54480d2320 2023-11-28T11:17:31.417 INFO:teuthology.orchestra.run.smithi157.stdout:4 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 msgr2=0x7f54481500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f543800c550 tx=0x7f543800c860 comp rx=0 tx=0).stop 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 shutdown_connections 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5448161340 0x7f54480b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54480d2320 0x7f54481500e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 --2- 172.21.15.157:0/1782536561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f544815c9b0 0x7f5448154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 >> 172.21.15.157:0/1782536561 conn(0x7f5448072600 msgr2=0x7f54480b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 shutdown_connections 2023-11-28T11:17:31.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.417+0000 7f544ddff700 1 -- 172.21.15.157:0/1782536561 wait complete. 2023-11-28T11:17:31.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new b7ea5e07-6301-45af-899d-a751ba02d3a1 5 2023-11-28T11:17:31.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.631+0000 7f9bb2d9d700 1 Processor -- start 2023-11-28T11:17:31.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.631+0000 7f9bb2d9d700 1 -- start start 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb2d9d700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0d0d20 con 0x7f9bac15c450 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb2d9d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0cf6b0 con 0x7f9bac0cffa0 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb2d9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0cdd80 con 0x7f9bac160af0 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb159a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9bac0cffa0 0x7f9bac0d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58502/0 (socket says 172.21.15.157:58502) 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb159a700 1 -- 172.21.15.157:0/988370657 learned_addr learned my addr 172.21.15.157:0/988370657 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.632+0000 7f9bb259c700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb259c700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb259c700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb259c700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac0cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb259c700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac0cf170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290273555 0 0) 0x7f9bac0cf6b0 con 0x7f9bac0cffa0 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b980035b0 con 0x7f9bac0cffa0 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680717445 0 0) 0x7f9bac0d0d20 con 0x7f9bac15c450 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac0cf6b0 con 0x7f9bac15c450 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 266860600 0 0) 0x7f9b980035b0 con 0x7f9bac0cffa0 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac0cf170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac0cf170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 >> v1:172.21.15.53:6789/0 conn(0x7f9bac15c450 legacy=0x7f9bac0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0cd500 con 0x7f9bac0cffa0 2023-11-28T11:17:31.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f9b9c0016b0 con 0x7f9bac0cffa0 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 >> v1:172.21.15.157:6789/0 conn(0x7f9bac0cffa0 legacy=0x7f9bac0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9b98004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b98005150 0x7f9b980037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b98003d30 0x7f9b980041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0cdd80 con 0x7f9b98005150 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9800bb50 con 0x7f9b98003d30 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.633+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9800bd10 con 0x7f9bac160af0 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9b98004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb259c700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b98003d30 0x7f9b980041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb159a700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b98005150 0x7f9b980037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b98003d30 msgr2=0x7f9b980041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b98003d30 0x7f9b980041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b98005150 msgr2=0x7f9b980037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b98005150 0x7f9b980037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 -- 172.21.15.157:0/988370657 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980035b0 con 0x7f9bac160af0 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9b98004bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8001940 tx=0x7f9b98005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6a761c313a2c52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.634+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8001430 con 0x7f9bac160af0 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba8001d70 con 0x7f9bac160af0 2023-11-28T11:17:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb0d99700 1 -- 172.21.15.157:0/988370657 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8005510 con 0x7f9bac160af0 2023-11-28T11:17:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9b98004bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 --2- 172.21.15.157:0/988370657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9b98004bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8001940 tx=0x7f9b98005880 comp rx=0 tx=0).stop 2023-11-28T11:17:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb259c700 1 -- 172.21.15.157:0/988370657 reap_dead start 2023-11-28T11:17:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/988370657 shutdown_connections 2023-11-28T11:17:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/988370657 >> 172.21.15.157:0/988370657 conn(0x7f9bac072630 msgr2=0x7f9bac06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/988370657 shutdown_connections 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.635+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/988370657 wait complete. 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.637+0000 7f9bb2d9d700 1 Processor -- start 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.637+0000 7f9bb2d9d700 1 -- start start 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 0x7f9bac1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac158f80 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac1553d0 con 0x7f9bac158b10 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb2d9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0ca120 con 0x7f9bac160af0 2023-11-28T11:17:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb1d9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38706/0 (socket says 172.21.15.157:38706) 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb159a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 0x7f9bac1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb259c700 1 -- 172.21.15.157:0/4144973671 learned_addr learned my addr 172.21.15.157:0/4144973671 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.638+0000 7f9bb1d9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:33950/0 (socket says 172.21.15.157:33950) 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb1d9b700 1 -- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb1d9b700 1 -- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb1d9b700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 -- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 msgr2=0x7f9bac154e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 -- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 msgr2=0x7f9bac0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 -- 172.21.15.157:0/4144973671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0cd7c0 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb259c700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb159a700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 0x7f9bac1585d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c001eb0 tx=0x7f9b9c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00f660 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c00f800 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac160f70 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.639+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00fbb0 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.640+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac0cdaa0 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.640+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9b9c01d460 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.640+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9b9c01dc70 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.641+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac061e80 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.644+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9b9c01d6f0 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.730+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b7ea5e07-6301-45af-899d-a751ba02d3a1", "id": 5} v 0) v1 -- 0x7f9bac0b53f0 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.733+0000 7f9ba27fc700 1 -- 172.21.15.157:0/4144973671 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b7ea5e07-6301-45af-899d-a751ba02d3a1", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b9c01d960 con 0x7f9bac0cb1c0 2023-11-28T11:17:31.734 INFO:teuthology.orchestra.run.smithi157.stdout:5 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 msgr2=0x7f9bac1585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 0x7f9bac1585d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c001eb0 tx=0x7f9b9c0075f0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 shutdown_connections 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac160af0 0x7f9bac154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac158b10 0x7f9bac0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 --2- 172.21.15.157:0/4144973671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bac0cb1c0 0x7f9bac1585d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 >> 172.21.15.157:0/4144973671 conn(0x7f9bac072630 msgr2=0x7f9bac06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 shutdown_connections 2023-11-28T11:17:31.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.734+0000 7f9bb2d9d700 1 -- 172.21.15.157:0/4144973671 wait complete. 2023-11-28T11:17:31.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 5a77ab3e-acb3-42e9-a245-99faff511a85 6 2023-11-28T11:17:31.888 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:31.888+0000 7fe4bbac8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:17:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.946+0000 7fdcd0bff700 1 Processor -- start 2023-11-28T11:17:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.946+0000 7fdcd0bff700 1 -- start start 2023-11-28T11:17:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcd0bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdccc0b20b0 0x7fdccc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcd0bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0b0230 con 0x7fdccc0d01b0 2023-11-28T11:17:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcd0bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0b4570 con 0x7fdccc0d1330 2023-11-28T11:17:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcd0bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc15cec0 con 0x7fdccc0b20b0 2023-11-28T11:17:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcc9d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdccc0d1330 0x7fdccc0b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58512/0 (socket says 172.21.15.157:58512) 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdcc9d9b700 1 -- 172.21.15.157:0/2282644826 learned_addr learned my addr 172.21.15.157:0/2282644826 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.947+0000 7fdccad9d700 1 --2- 172.21.15.157:0/2282644826 >> v2:172.21.15.157:3300/0 conn(0x7fdccc0b20b0 0x7fdccc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338917857 0 0) 0x7fdccc0b4570 con 0x7fdccc0d1330 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb40035b0 con 0x7fdccc0d1330 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdccad9d700 1 -- 172.21.15.157:0/2282644826 >> v1:172.21.15.157:6789/0 conn(0x7fdccc0d1330 legacy=0x7fdccc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdccad9d700 1 -- 172.21.15.157:0/2282644826 >> v1:172.21.15.53:6789/0 conn(0x7fdccc0d01b0 legacy=0x7fdccc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324253763 0 0) 0x7fdccc0b0230 con 0x7fdccc0d01b0 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdccad9d700 1 -- 172.21.15.157:0/2282644826 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0b4850 con 0x7fdccc0b20b0 2023-11-28T11:17:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdccad9d700 1 --2- 172.21.15.157:0/2282644826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0b20b0 0x7fdccc0b0dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102965ccdb0fa67d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.948+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400e070 con 0x7fdccc0b20b0 2023-11-28T11:17:31.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc400bb20 con 0x7fdccc0b20b0 2023-11-28T11:17:31.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcc959a700 1 -- 172.21.15.157:0/2282644826 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400bed0 con 0x7fdccc0b20b0 2023-11-28T11:17:31.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 -- 172.21.15.157:0/2282644826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0b20b0 msgr2=0x7fdccc0b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/2282644826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0b20b0 0x7fdccc0b0dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc400b330 comp rx=0 tx=0).stop 2023-11-28T11:17:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 -- 172.21.15.157:0/2282644826 shutdown_connections 2023-11-28T11:17:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/2282644826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0b20b0 0x7fdccc0b0dd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 -- 172.21.15.157:0/2282644826 >> 172.21.15.157:0/2282644826 conn(0x7fdccc09c3f0 msgr2=0x7fdccc0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 -- 172.21.15.157:0/2282644826 shutdown_connections 2023-11-28T11:17:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.949+0000 7fdcd0bff700 1 -- 172.21.15.157:0/2282644826 wait complete. 2023-11-28T11:17:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.952+0000 7fdcd0bff700 1 Processor -- start 2023-11-28T11:17:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.952+0000 7fdcd0bff700 1 -- start start 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc0d01b0 0x7fdccc14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc159a10 con 0x7fdccc0d01b0 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0ce130 con 0x7fdccc0d1330 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcd0bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cce30 con 0x7fdccc0b20b0 2023-11-28T11:17:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdccad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdccad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38728/0 (socket says 172.21.15.157:38728) 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdccad9d700 1 -- 172.21.15.157:0/3592576136 learned_addr learned my addr 172.21.15.157:0/3592576136 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcc9d9b700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc0d01b0 0x7fdccc14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcca59c700 1 -- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 msgr2=0x7fdccc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcca59c700 1 -- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 msgr2=0x7fdccc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcca59c700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdcca59c700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.954+0000 7fdccad9d700 1 -- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 msgr2=0x7fdccc0cd8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdccad9d700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdccad9d700 1 -- 172.21.15.157:0/3592576136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc0d01b0 msgr2=0x7fdccc14a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdccad9d700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc0d01b0 0x7fdccc14a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdccad9d700 1 -- 172.21.15.157:0/3592576136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4009a70 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdccad9d700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4014040 tx=0x7fdcc4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401b810 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4020070 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401bbb0 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc0afec0 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc0afbe0 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.955+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fdcc402e070 con 0x7fdccc0d1330 2023-11-28T11:17:31.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.956+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fdcc4022070 con 0x7fdccc0d1330 2023-11-28T11:17:31.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.956+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc0b2600 con 0x7fdccc0d1330 2023-11-28T11:17:31.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:31.959+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fdcc402c060 con 0x7fdccc0d1330 2023-11-28T11:17:31.974 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:31.973+0000 7f09d56c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:17:31.992 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:31.992+0000 7fe4bbac8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:17:32.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.047+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a77ab3e-acb3-42e9-a245-99faff511a85", "id": 6} v 0) v1 -- 0x7fdccc0ce300 con 0x7fdccc0d1330 2023-11-28T11:17:32.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.051+0000 7fdcbaffd700 1 -- 172.21.15.157:0/3592576136 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a77ab3e-acb3-42e9-a245-99faff511a85", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdccc0ce300 con 0x7fdccc0d1330 2023-11-28T11:17:32.052 INFO:teuthology.orchestra.run.smithi157.stdout:6 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 msgr2=0x7fdccc0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4014040 tx=0x7fdcc4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 shutdown_connections 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdccc0b20b0 0x7fdccc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdccc0d1330 0x7fdccc0ce5e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 --2- 172.21.15.157:0/3592576136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc0d01b0 0x7fdccc14a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 >> 172.21.15.157:0/3592576136 conn(0x7fdccc09c3f0 msgr2=0x7fdccc06b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 shutdown_connections 2023-11-28T11:17:32.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.052+0000 7fdcd0bff700 1 -- 172.21.15.157:0/3592576136 wait complete. 2023-11-28T11:17:32.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 4e5d1ceb-346f-40a5-8510-dc9611249e8a 7 2023-11-28T11:17:32.078 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:32.077+0000 7f09d56c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:17:32.178 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:32.177+0000 7fe4bbac8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:17:32.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.263+0000 7feb349ff700 1 Processor -- start 2023-11-28T11:17:32.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.263+0000 7feb349ff700 1 -- start start 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.264+0000 7feb349ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.264+0000 7feb349ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb300cd7c0 0x7feb300cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.264+0000 7feb349ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300ce160 con 0x7feb30161340 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.264+0000 7feb349ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300b5800 con 0x7feb300cd7c0 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.264+0000 7feb349ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300d1950 con 0x7feb3015c9b0 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2e59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb27fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feb30161340 0x7feb300d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58528/0 (socket says 172.21.15.157:58528) 2023-11-28T11:17:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb27fff700 1 -- 172.21.15.157:0/2323680494 learned_addr learned my addr 172.21.15.157:0/2323680494 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-28T11:17:32.267 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:32.265+0000 7f09d56c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/2323680494 >> v2:172.21.15.157:3300/0 conn(0x7feb300cd7c0 0x7feb300cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2e59c700 1 -- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 msgr2=0x7feb300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2e59c700 1 -- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 msgr2=0x7feb300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2e59c700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2e59c700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2dd9b700 1 -- 172.21.15.157:0/2323680494 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602833400 0 0) 0x7feb300ce160 con 0x7feb30161340 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2dd9b700 1 -- 172.21.15.157:0/2323680494 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb1c0035b0 con 0x7feb30161340 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2ed9d700 1 -- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 msgr2=0x7feb300b4bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2ed9d700 1 -- 172.21.15.157:0/2323680494 >> v1:172.21.15.157:6789/0 conn(0x7feb30161340 legacy=0x7feb300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.265+0000 7feb2ed9d700 1 -- 172.21.15.157:0/2323680494 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300d24f0 con 0x7feb300cd7c0 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.266+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb300cd7c0 0x7feb300cdc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb28009d90 tx=0x7feb2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269761881eab7432 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.266+0000 7feb2dd9b700 1 -- 172.21.15.157:0/2323680494 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800e070 con 0x7feb300cd7c0 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.266+0000 7feb2dd9b700 1 -- 172.21.15.157:0/2323680494 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2800bb20 con 0x7feb300cd7c0 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.266+0000 7feb2dd9b700 1 -- 172.21.15.157:0/2323680494 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28012430 con 0x7feb300cd7c0 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.266+0000 7feb349ff700 1 -- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb300cd7c0 msgr2=0x7feb300cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb300cd7c0 0x7feb300cdc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb28009d90 tx=0x7feb2800b330 comp rx=0 tx=0).stop 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 -- 172.21.15.157:0/2323680494 shutdown_connections 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb300cd7c0 0x7feb300cdc20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 --2- 172.21.15.157:0/2323680494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3015c9b0 0x7feb300b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 -- 172.21.15.157:0/2323680494 >> 172.21.15.157:0/2323680494 conn(0x7feb30072600 msgr2=0x7feb300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 -- 172.21.15.157:0/2323680494 shutdown_connections 2023-11-28T11:17:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.267+0000 7feb349ff700 1 -- 172.21.15.157:0/2323680494 wait complete. 2023-11-28T11:17:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.268+0000 7feb349ff700 1 Processor -- start 2023-11-28T11:17:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.269+0000 7feb349ff700 1 -- start start 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.269+0000 7feb349ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cd7c0 0x7feb30150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.269+0000 7feb349ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb349ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb349ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300cc050 con 0x7feb300cd7c0 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb349ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb30150f40 con 0x7feb3015c9b0 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb349ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300ca000 con 0x7feb30161340 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38744/0 (socket says 172.21.15.157:38744) 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 -- 172.21.15.157:0/3798274655 learned_addr learned my addr 172.21.15.157:0/3798274655 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-28T11:17:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2e59c700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cd7c0 0x7feb30150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2ed9d700 1 -- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 msgr2=0x7feb300c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2ed9d700 1 -- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 msgr2=0x7feb300c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb2ed9d700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 -- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 msgr2=0x7feb300c9980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 -- 172.21.15.157:0/3798274655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cd7c0 msgr2=0x7feb30150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cd7c0 0x7feb30150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 -- 172.21.15.157:0/3798274655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28009a70 con 0x7feb3015c9b0 2023-11-28T11:17:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.270+0000 7feb27fff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb18001eb0 tx=0x7feb1800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18005d70 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb180070d0 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18007460 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb300d1300 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb300b5520 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7feb18019030 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.271+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7feb180075c0 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.272+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb300d15e0 con 0x7feb3015c9b0 2023-11-28T11:17:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.274+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7feb18017070 con 0x7feb3015c9b0 2023-11-28T11:17:32.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.360+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4e5d1ceb-346f-40a5-8510-dc9611249e8a", "id": 7} v 0) v1 -- 0x7feb300745c0 con 0x7feb3015c9b0 2023-11-28T11:17:32.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.363+0000 7feb26ffd700 1 -- 172.21.15.157:0/3798274655 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e5d1ceb-346f-40a5-8510-dc9611249e8a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7feb18007780 con 0x7feb3015c9b0 2023-11-28T11:17:32.365 INFO:teuthology.orchestra.run.smithi157.stdout:7 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.364+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 msgr2=0x7feb300c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.364+0000 7feb349ff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb18001eb0 tx=0x7feb1800c7b0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 shutdown_connections 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb30161340 0x7feb300c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cd7c0 0x7feb30150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 --2- 172.21.15.157:0/3798274655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3015c9b0 0x7feb300c9440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 >> 172.21.15.157:0/3798274655 conn(0x7feb30072600 msgr2=0x7feb30160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 shutdown_connections 2023-11-28T11:17:32.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-28T11:17:32.365+0000 7feb349ff700 1 -- 172.21.15.157:0/3798274655 wait complete. 2023-11-28T11:17:32.383 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:17:32.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:17:32.388 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:17:32.388 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:17:32.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:17:32.392 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:17:32.392 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:17:32.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:17:32.398 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:17:32.398 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:17:32.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:17:32.403 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:17:32.403 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:17:32.403 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:17:32.428 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:17:32.428 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:17:32.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:17:32.432 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:17:32.432 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:17:32.432 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:17:32.437 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:17:32.437 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:17:32.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:17:32.442 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:17:32.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:32.633 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:32.632+0000 7fe4bbac8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:17:32.697 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:32.696+0000 7f09d56c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:17:32.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.724+0000 7fc200fff700 1 Processor -- start 2023-11-28T11:17:32.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.725+0000 7fc200fff700 1 -- start start 2023-11-28T11:17:32.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc200fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1fc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc200fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0d1950 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc200fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0ce140 con 0x7fc1fc161340 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc200fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b5800 con 0x7fc1fc0cd7c0 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fa59c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc1fc15c9b0 0x7fc1fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33554/0 (socket says 172.21.15.53:33554) 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fa59c700 1 -- 172.21.15.53:0/3362541017 learned_addr learned my addr 172.21.15.53:0/3362541017 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1fc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 msgr2=0x7fc1fc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546847905 0 0) 0x7fc1fc0d1950 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 msgr2=0x7fc1fc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1fc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e80035b0 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.727+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1fc0cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936477029 0 0) 0x7fc1fc0ce140 con 0x7fc1fc161340 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1fc0d1950 con 0x7fc1fc161340 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3774391874 0 0) 0x7fc1e80035b0 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 msgr2=0x7fc1fc0cdc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1fc0cdc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 >> v1:172.21.15.157:6789/0 conn(0x7fc1fc161340 legacy=0x7fc1fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0d24f0 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fc1ec001670 con 0x7fc1fc15c9b0 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 >> v1:172.21.15.53:6789/0 conn(0x7fc1fc15c9b0 legacy=0x7fc1fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e8003bc0 0x7fc1e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e8004d90 0x7fc1e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b5800 con 0x7fc1e8004d90 2023-11-28T11:17:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800bc50 con 0x7fc1e8003bc0 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800be10 con 0x7fc1fc0cd7c0 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1fa59c700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e8003bc0 0x7fc1e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e8004d90 0x7fc1e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.728+0000 7fc1f3fff700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 msgr2=0x7fc1e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1fc0cd7c0 0x7fc1e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e8003bc0 msgr2=0x7fc1e8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e8003bc0 0x7fc1e8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80035b0 con 0x7fc1e8004d90 2023-11-28T11:17:32.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e8004d90 0x7fc1e8005200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009fd0 tx=0x7fc1e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebafabbb85ac5a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:32.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4007e20 con 0x7fc1e8004d90 2023-11-28T11:17:32.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4003a20 con 0x7fc1e8004d90 2023-11-28T11:17:32.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.729+0000 7fc1f9d9b700 1 -- 172.21.15.53:0/3362541017 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4003dd0 con 0x7fc1e8004d90 2023-11-28T11:17:32.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 -- 172.21.15.53:0/3362541017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e8004d90 msgr2=0x7fc1e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 --2- 172.21.15.53:0/3362541017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e8004d90 0x7fc1e8005200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009fd0 tx=0x7fc1e8005880 comp rx=0 tx=0).stop 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc1fad9d700 1 -- 172.21.15.53:0/3362541017 reap_dead start 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 -- 172.21.15.53:0/3362541017 shutdown_connections 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 -- 172.21.15.53:0/3362541017 >> 172.21.15.53:0/3362541017 conn(0x7fc1fc072600 msgr2=0x7fc1fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 -- 172.21.15.53:0/3362541017 shutdown_connections 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.730+0000 7fc200fff700 1 -- 172.21.15.53:0/3362541017 wait complete. 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.732+0000 7fc200fff700 1 Processor -- start 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.732+0000 7fc200fff700 1 -- start start 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1fc15e480 0x7fc1fc15e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc15ee60 con 0x7fc1fc0cb880 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1f6520 con 0x7fc1fc15e480 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc200fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1f6800 con 0x7fc1e8004d90 2023-11-28T11:17:32.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc1f3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc1fa59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc1f3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49728/0 (socket says 172.21.15.53:49728) 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc1f3fff700 1 -- 172.21.15.53:0/3455518125 learned_addr learned my addr 172.21.15.53:0/3455518125 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.733+0000 7fc1fad9d700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1fc15e480 0x7fc1fc15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1fa59c700 1 -- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 msgr2=0x7fc1fc1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1fa59c700 1 -- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 msgr2=0x7fc1fc1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1fa59c700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1fa59c700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 -- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 msgr2=0x7fc1fc1f5cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 -- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1fc15e480 msgr2=0x7fc1fc15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1fc15e480 0x7fc1fc15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 -- 172.21.15.53:0/3455518125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0d1300 con 0x7fc1fc0cb880 2023-11-28T11:17:32.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f3fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400f7c0 tx=0x7fc1e400fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4001500 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40052e0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4003680 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.734+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc0b5520 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.735+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc0d15e0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.735+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc1e4003af0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.735+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc1e401dd20 con 0x7fc1fc0cb880 2023-11-28T11:17:32.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.736+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc0745c0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.738+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc1e401d510 con 0x7fc1fc0cb880 2023-11-28T11:17:32.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.813+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1fc0b5fa0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.814+0000 7fc1f2ffd700 1 -- 172.21.15.53:0/3455518125 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc1e401d6b0 con 0x7fc1fc0cb880 2023-11-28T11:17:32.815 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:32.815 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:32.362212+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:32.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 msgr2=0x7fc1fc14d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:32.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400f7c0 tx=0x7fc1e400fad0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 shutdown_connections 2023-11-28T11:17:32.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1e8004d90 0x7fc1fc1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1fc15e480 0x7fc1fc15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 --2- 172.21.15.53:0/3455518125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1fc0cb880 0x7fc1fc14d520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 >> 172.21.15.53:0/3455518125 conn(0x7fc1fc072600 msgr2=0x7fc1fc0b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 shutdown_connections 2023-11-28T11:17:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:32.815+0000 7fc200fff700 1 -- 172.21.15.53:0/3455518125 wait complete. 2023-11-28T11:17:32.819 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:32.818+0000 7fe4bbac8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:17:32.835 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:17:32.835 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:17:32.865 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:32.864+0000 7f09d56c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:17:33.108 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:33.107+0000 7fe4bbac8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:17:33.124 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-11-28T11:17:33.124+0000 7f4c4ed41640 -1 Falling back to public interface 2023-11-28T11:17:33.126 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-11-28T11:17:33.125+0000 7f02e55e3640 -1 Falling back to public interface 2023-11-28T11:17:33.128 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-11-28T11:17:33.128+0000 7faab48db640 -1 Falling back to public interface 2023-11-28T11:17:33.132 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-11-28T11:17:33.131+0000 7fcb0ef65640 -1 Falling back to public interface 2023-11-28T11:17:33.136 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-11-28T11:17:33.135+0000 7f55be713640 -1 Falling back to public interface 2023-11-28T11:17:33.139 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-11-28T11:17:33.138+0000 7f2afc30c640 -1 Falling back to public interface 2023-11-28T11:17:33.145 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-11-28T11:17:33.144+0000 7f58d5f7f640 -1 Falling back to public interface 2023-11-28T11:17:33.148 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:33.147+0000 7f09d56c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:17:33.180 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-11-28T11:17:33.179+0000 7f9c10ee6640 -1 Falling back to public interface 2023-11-28T11:17:33.322 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:33.322+0000 7fe4bbac8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:17:33.350 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:33.349+0000 7f09d56c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:17:33.479 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:33.478+0000 7fe4bbac8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:17:33.499 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:33.498+0000 7f09d56c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:17:33.640 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:33.819 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.818+0000 7fbfc93ff700 1 Processor -- start 2023-11-28T11:17:33.819 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.818+0000 7fbfc93ff700 1 -- start start 2023-11-28T11:17:33.821 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.819+0000 7fbfc93ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 0x7fbfc40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.819+0000 7fbfc93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.819+0000 7fbfc93ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40d1950 con 0x7fbfc4161340 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.819+0000 7fbfc93ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc40b5800 con 0x7fbfc415c9b0 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.819+0000 7fbfc93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40ce140 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc27fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbfc4161340 0x7fbfc40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33568/0 (socket says 172.21.15.53:33568) 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc27fc700 1 -- 172.21.15.53:0/3387979581 learned_addr learned my addr 172.21.15.53:0/3387979581 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3387979581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc2ffd700 1 --2- 172.21.15.53:0/3387979581 >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 0x7fbfc40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc1ffb700 1 -- 172.21.15.53:0/3387979581 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211469840 0 0) 0x7fbfc40d1950 con 0x7fbfc4161340 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc1ffb700 1 -- 172.21.15.53:0/3387979581 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfac0035b0 con 0x7fbfc4161340 2023-11-28T11:17:33.822 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3387979581 >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 msgr2=0x7fbfc40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3387979581 >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 0x7fbfc40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3387979581 >> v1:172.21.15.53:6789/0 conn(0x7fbfc4161340 legacy=0x7fbfc40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3387979581 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40d24f0 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.820+0000 7fbfc2ffd700 1 --2- 172.21.15.53:0/3387979581 >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 0x7fbfc40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3387979581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc40cdc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009d90 tx=0x7fbfbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2094b53d83bb80ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc1ffb700 1 -- 172.21.15.53:0/3387979581 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00e070 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc1ffb700 1 -- 172.21.15.53:0/3387979581 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00bb20 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.823 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc1ffb700 1 -- 172.21.15.53:0/3387979581 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc012430 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3387979581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 msgr2=0x7fbfc40cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3387979581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc40cdc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009d90 tx=0x7fbfbc00b330 comp rx=0 tx=0).stop 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3387979581 shutdown_connections 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3387979581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc40cdc00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3387979581 >> v2:172.21.15.157:3300/0 conn(0x7fbfc415c9b0 0x7fbfc40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.821+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3387979581 >> 172.21.15.53:0/3387979581 conn(0x7fbfc4072600 msgr2=0x7fbfc40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:33.824 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.822+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3387979581 shutdown_connections 2023-11-28T11:17:33.825 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.822+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3387979581 wait complete. 2023-11-28T11:17:33.825 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.823+0000 7fbfc93ff700 1 Processor -- start 2023-11-28T11:17:33.825 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.823+0000 7fbfc93ff700 1 -- start start 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc41576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc415c9b0 0x7fbfc41e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41e6e20 con 0x7fbfc4161340 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41e7100 con 0x7fbfc415c9b0 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.824+0000 7fbfc93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41e73e0 con 0x7fbfc40cd7c0 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49732/0 (socket says 172.21.15.53:49732) 2023-11-28T11:17:33.827 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc415c9b0 0x7fbfc41e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3393110353 learned_addr learned my addr 172.21.15.53:0/3393110353 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc2ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc41576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 msgr2=0x7fbfc41576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc41576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc415c9b0 msgr2=0x7fbfc41e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc415c9b0 0x7fbfc41e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 -- 172.21.15.53:0/3393110353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40094a0 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.825+0000 7fbfc37fe700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc014040 tx=0x7fbfbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0129f0 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc020070 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc012db0 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc009a70 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc40d1300 con 0x7fbfc4161340 2023-11-28T11:17:33.828 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fbfbc02e070 con 0x7fbfc4161340 2023-11-28T11:17:33.829 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.826+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fbfbc022070 con 0x7fbfc4161340 2023-11-28T11:17:33.829 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.827+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc40b5520 con 0x7fbfc4161340 2023-11-28T11:17:33.830 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.830+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbfbc02c060 con 0x7fbfc4161340 2023-11-28T11:17:33.887 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:33.887+0000 7fe4bbac8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:17:33.896 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:33.895+0000 7f09d56c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:17:33.905 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.904+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfc40d15e0 con 0x7fbfc4161340 2023-11-28T11:17:33.906 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.905+0000 7fbfab7fe700 1 -- 172.21.15.53:0/3393110353 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fbfbc0043e0 con 0x7fbfc4161340 2023-11-28T11:17:33.906 INFO:teuthology.misc.health.smithi053.stdout: 2023-11-28T11:17:33.906 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":10,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:32.362212+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 msgr2=0x7fbfc41e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc014040 tx=0x7fbfbc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 shutdown_connections 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfc40cd7c0 0x7fbfc41576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4161340 0x7fbfc41e78b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 --2- 172.21.15.53:0/3393110353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc415c9b0 0x7fbfc41e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:33.907 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 >> 172.21.15.53:0/3393110353 conn(0x7fbfc4072600 msgr2=0x7fbfc40b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:33.908 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 shutdown_connections 2023-11-28T11:17:33.908 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:33.906+0000 7fbfc93ff700 1 -- 172.21.15.53:0/3393110353 wait complete. 2023-11-28T11:17:33.921 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:17:34.021 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:34.020+0000 7fe4bbac8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:17:34.027 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:34.026+0000 7f09d56c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:17:34.261 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:34.260+0000 7fe4bbac8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:17:34.261 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:34.260+0000 7f09d56c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:17:34.916 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:34.915+0000 7f09d56c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:17:34.933 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-11-28T11:17:34.932+0000 7fe4bbac8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:17:35.944 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:35.942+0000 7f09889fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:17:35.944 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-11-28T11:17:35.943+0000 7f09889fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:17:36.633 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-11-28T11:17:36.632+0000 7f4c4ed41640 -1 osd.1 0 log_to_monitors true 2023-11-28T11:17:36.695 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-11-28T11:17:36.694+0000 7f02e55e3640 -1 osd.2 0 log_to_monitors true 2023-11-28T11:17:36.700 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-11-28T11:17:36.698+0000 7f55be713640 -1 osd.3 0 log_to_monitors true 2023-11-28T11:17:36.705 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-11-28T11:17:36.704+0000 7faab48db640 -1 osd.0 0 log_to_monitors true 2023-11-28T11:17:36.727 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-11-28T11:17:36.726+0000 7f58d5f7f640 -1 osd.6 0 log_to_monitors true 2023-11-28T11:17:36.773 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-11-28T11:17:36.772+0000 7f9c10ee6640 -1 osd.7 0 log_to_monitors true 2023-11-28T11:17:36.930 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-11-28T11:17:36.928+0000 7fcb0ef65640 -1 osd.4 0 log_to_monitors true 2023-11-28T11:17:36.985 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-11-28T11:17:36.984+0000 7f2afc30c640 -1 osd.5 0 log_to_monitors true 2023-11-28T11:17:39.432 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-11-28T11:17:39.430+0000 7f9bff5f6700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:17:39.446 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-11-28T11:17:39.445+0000 7f9bf75fb700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.448 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-11-28T11:17:39.447+0000 7fcafd5f6700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:17:39.467 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-11-28T11:17:39.466+0000 7f58c45f6700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:17:39.471 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-11-28T11:17:39.470+0000 7fcaf55fb700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.485 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-11-28T11:17:39.484+0000 7f58bc5fb700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.683 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-11-28T11:17:39.681+0000 7f55ae5f9700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:17:39.704 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-11-28T11:17:39.703+0000 7f55a4dfb700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.709 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-11-28T11:17:39.709+0000 7f4c3d3f6700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:17:39.723 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-11-28T11:17:39.722+0000 7faaa2ff6700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:17:39.725 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-11-28T11:17:39.724+0000 7f4c353fb700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.738 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-11-28T11:17:39.737+0000 7faa9affb700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:39.762 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-11-28T11:17:39.762+0000 7f02d3bf6700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:17:39.778 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-11-28T11:17:39.777+0000 7f02cbbfb700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:40.179 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-11-28T11:17:40.178+0000 7f2aea9f6700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:17:40.196 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-11-28T11:17:40.195+0000 7f2ae29fb700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:17:40.727 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:40.915 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.914+0000 7f13dd3ff700 1 Processor -- start 2023-11-28T11:17:40.915 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.914+0000 7f13dd3ff700 1 -- start start 2023-11-28T11:17:40.916 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.915+0000 7f13dd3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13d815c9b0 0x7f13d80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:40.916 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.915+0000 7f13dd3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80d1950 con 0x7f13d80cd7c0 2023-11-28T11:17:40.916 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.915+0000 7f13dd3ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80d24f0 con 0x7f13d8161340 2023-11-28T11:17:40.917 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.915+0000 7f13dd3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80b5800 con 0x7f13d815c9b0 2023-11-28T11:17:40.917 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.915+0000 7f13d77fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f13d80cd7c0 0x7f13d80b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45184/0 (socket says 172.21.15.53:45184) 2023-11-28T11:17:40.917 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d77fe700 1 -- 172.21.15.53:0/755261659 learned_addr learned my addr 172.21.15.53:0/755261659 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:40.917 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d6ffd700 1 --2- 172.21.15.53:0/755261659 >> v2:172.21.15.53:3300/0 conn(0x7f13d815c9b0 0x7f13d80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:40.918 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d6ffd700 1 -- 172.21.15.53:0/755261659 >> v1:172.21.15.157:6789/0 conn(0x7f13d8161340 legacy=0x7f13d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:40.918 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d6ffd700 1 -- 172.21.15.53:0/755261659 >> v1:172.21.15.53:6789/0 conn(0x7f13d80cd7c0 legacy=0x7f13d80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:40.918 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d6ffd700 1 -- 172.21.15.53:0/755261659 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80b5520 con 0x7f13d815c9b0 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d6ffd700 1 --2- 172.21.15.53:0/755261659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d815c9b0 0x7f13d80b4bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009840 tx=0x7f13c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ec8f43edb3fe7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d5ffb700 1 -- 172.21.15.53:0/755261659 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8011830 con 0x7f13d815c9b0 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.916+0000 7f13d5ffb700 1 -- 172.21.15.53:0/755261659 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8011990 con 0x7f13d815c9b0 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13d5ffb700 1 -- 172.21.15.53:0/755261659 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8011d20 con 0x7f13d815c9b0 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 -- 172.21.15.53:0/755261659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d815c9b0 msgr2=0x7f13d80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/755261659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d815c9b0 0x7f13d80b4bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009840 tx=0x7f13c8009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 -- 172.21.15.53:0/755261659 shutdown_connections 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/755261659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d815c9b0 0x7f13d80b4bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 -- 172.21.15.53:0/755261659 >> 172.21.15.53:0/755261659 conn(0x7f13d8072600 msgr2=0x7f13d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 -- 172.21.15.53:0/755261659 shutdown_connections 2023-11-28T11:17:40.919 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.917+0000 7f13dd3ff700 1 -- 172.21.15.53:0/755261659 wait complete. 2023-11-28T11:17:40.920 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.920+0000 7f13dd3ff700 1 Processor -- start 2023-11-28T11:17:40.920 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.920+0000 7f13dd3ff700 1 -- start start 2023-11-28T11:17:40.923 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:40.923 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d815c9b0 0x7f13d815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:40.923 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d8161340 0x7f13d8153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8155740 con 0x7f13d80cd7c0 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8155990 con 0x7f13d8161340 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13dd3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8155c70 con 0x7f13d815c9b0 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13d6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13d6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39454/0 (socket says 172.21.15.53:39454) 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13d6ffd700 1 -- 172.21.15.53:0/3509411552 learned_addr learned my addr 172.21.15.53:0/3509411552 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13d67fc700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d815c9b0 0x7f13d815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.921+0000 7f13d77fe700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d8161340 0x7f13d8153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 -- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d815c9b0 msgr2=0x7f13d815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d815c9b0 0x7f13d815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 -- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d8161340 msgr2=0x7f13d8153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:40.924 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d8161340 0x7f13d8153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 -- 172.21.15.53:0/3509411552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80094a0 con 0x7f13d80cd7c0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13d6ffd700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80038a0 con 0x7f13d80cd7c0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c801e070 con 0x7f13d80cd7c0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80d15e0 con 0x7f13d80cd7c0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8003c60 con 0x7f13d80cd7c0 2023-11-28T11:17:40.925 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.922+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d80d1300 con 0x7f13d80cd7c0 2023-11-28T11:17:40.926 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.923+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d80745c0 con 0x7f13d80cd7c0 2023-11-28T11:17:40.926 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.924+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f13c800f050 con 0x7f13d80cd7c0 2023-11-28T11:17:40.926 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.925+0000 7f13bf7fe700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 0x7f13c406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:40.926 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.925+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f13c8086870 con 0x7f13d80cd7c0 2023-11-28T11:17:40.926 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.925+0000 7f13d67fc700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 0x7f13c406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:40.927 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.926+0000 7f13d67fc700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 0x7f13c406a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13d8158300 tx=0x7f13cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:40.930 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:40.930+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13c8053660 con 0x7f13d80cd7c0 2023-11-28T11:17:41.136 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.135+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13d80b50b0 con 0x7f13d80cd7c0 2023-11-28T11:17:41.137 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.136+0000 7f13bf7fe700 1 -- 172.21.15.53:0/3509411552 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10536 (secure 0 0 0) 0x7f13c8015090 con 0x7f13d80cd7c0 2023-11-28T11:17:41.137 INFO:teuthology.misc.health.smithi053.stdout: 2023-11-28T11:17:41.137 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":14,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:40.436721+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:41.139 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 msgr2=0x7f13c406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 0x7f13c406a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13d8158300 tx=0x7f13cc009040 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 msgr2=0x7f13d8153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8004750 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 shutdown_connections 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f13c40682a0 0x7f13c406a750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d8161340 0x7f13d8153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13d80cd7c0 0x7f13d8153f30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 --2- 172.21.15.53:0/3509411552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d815c9b0 0x7f13d815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 >> 172.21.15.53:0/3509411552 conn(0x7f13d8072600 msgr2=0x7f13d815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 shutdown_connections 2023-11-28T11:17:41.140 INFO:teuthology.misc.health.smithi053.stderr:2023-11-28T11:17:41.138+0000 7f13dd3ff700 1 -- 172.21.15.53:0/3509411552 wait complete. 2023-11-28T11:17:41.155 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:17:41.155 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:17:41.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:17:41.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.349+0000 7fa4593ff700 1 Processor -- start 2023-11-28T11:17:41.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.349+0000 7fa4593ff700 1 -- start start 2023-11-28T11:17:41.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4593ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa45415c9b0 0x7fa4540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4593ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4540cd7c0 0x7fa4540cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4593ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540d1950 con 0x7fa454161340 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4593ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540b5800 con 0x7fa45415c9b0 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4593ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540ce140 con 0x7fa4540cd7c0 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa452ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa45415c9b0 0x7fa4540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.350+0000 7fa4527fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa454161340 0x7fa4540d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45196/0 (socket says 172.21.15.53:45196) 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa4527fc700 1 -- 172.21.15.53:0/1689449480 learned_addr learned my addr 172.21.15.53:0/1689449480 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa45415c9b0 0x7fa4540b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39456/0 (socket says 172.21.15.53:39456) 2023-11-28T11:17:41.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa4537fe700 1 --2- 172.21.15.53:0/1689449480 >> v2:172.21.15.157:3300/0 conn(0x7fa4540cd7c0 0x7fa4540cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 -- 172.21.15.53:0/1689449480 >> v2:172.21.15.157:3300/0 conn(0x7fa4540cd7c0 msgr2=0x7fa4540cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:41.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 --2- 172.21.15.53:0/1689449480 >> v2:172.21.15.157:3300/0 conn(0x7fa4540cd7c0 0x7fa4540cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 -- 172.21.15.53:0/1689449480 >> v1:172.21.15.53:6789/0 conn(0x7fa454161340 legacy=0x7fa4540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 -- 172.21.15.53:0/1689449480 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540d24f0 con 0x7fa45415c9b0 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa451ffb700 1 -- 172.21.15.53:0/1689449480 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826757492 0 0) 0x7fa4540d1950 con 0x7fa454161340 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa452ffd700 1 --2- 172.21.15.53:0/1689449480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa45415c9b0 0x7fa4540b4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa45406c170 tx=0x7fa444009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb9bb88a2aac487 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa451ffb700 1 -- 172.21.15.53:0/1689449480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011830 con 0x7fa45415c9b0 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa451ffb700 1 -- 172.21.15.53:0/1689449480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444011990 con 0x7fa45415c9b0 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.351+0000 7fa451ffb700 1 -- 172.21.15.53:0/1689449480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011d20 con 0x7fa45415c9b0 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 -- 172.21.15.53:0/1689449480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa45415c9b0 msgr2=0x7fa4540b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 --2- 172.21.15.53:0/1689449480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa45415c9b0 0x7fa4540b4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa45406c170 tx=0x7fa444009b20 comp rx=0 tx=0).stop 2023-11-28T11:17:41.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 -- 172.21.15.53:0/1689449480 shutdown_connections 2023-11-28T11:17:41.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 --2- 172.21.15.53:0/1689449480 >> v2:172.21.15.157:3300/0 conn(0x7fa4540cd7c0 0x7fa4540cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 --2- 172.21.15.53:0/1689449480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa45415c9b0 0x7fa4540b4bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 -- 172.21.15.53:0/1689449480 >> 172.21.15.53:0/1689449480 conn(0x7fa454072600 msgr2=0x7fa4540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:41.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 -- 172.21.15.53:0/1689449480 shutdown_connections 2023-11-28T11:17:41.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.352+0000 7fa4593ff700 1 -- 172.21.15.53:0/1689449480 wait complete. 2023-11-28T11:17:41.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.354+0000 7fa4593ff700 1 Processor -- start 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.354+0000 7fa4593ff700 1 -- start start 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 0x7fa4541e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 0x7fa4541e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415d350 con 0x7fa4540cd7c0 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540c7750 con 0x7fa454161340 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4593ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415d5a0 con 0x7fa45415c9b0 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa452ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa452ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39462/0 (socket says 172.21.15.53:39462) 2023-11-28T11:17:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa452ffd700 1 -- 172.21.15.53:0/885022463 learned_addr learned my addr 172.21.15.53:0/885022463 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4527fc700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 0x7fa4541e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.355+0000 7fa4537fe700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 0x7fa4541e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 -- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 msgr2=0x7fa4541e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 0x7fa4541e8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 -- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 msgr2=0x7fa4541e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 0x7fa4541e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 -- 172.21.15.53:0/885022463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440094a0 con 0x7fa4540cd7c0 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa4527fc700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 0x7fa4541e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa4537fe700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 0x7fa4541e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:41.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa452ffd700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa444014040 tx=0x7fa444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440038a0 con 0x7fa4540cd7c0 2023-11-28T11:17:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44401e070 con 0x7fa4540cd7c0 2023-11-28T11:17:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003c20 con 0x7fa4540cd7c0 2023-11-28T11:17:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4540d1300 con 0x7fa4540cd7c0 2023-11-28T11:17:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.356+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4540b5520 con 0x7fa4540cd7c0 2023-11-28T11:17:41.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.357+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4540d15e0 con 0x7fa4540cd7c0 2023-11-28T11:17:41.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.358+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fa44400f050 con 0x7fa4540cd7c0 2023-11-28T11:17:41.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.359+0000 7fa43b7fe700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 0x7fa44006a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:41.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.359+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fa444086590 con 0x7fa4540cd7c0 2023-11-28T11:17:41.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.360+0000 7fa4527fc700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 0x7fa44006a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.360+0000 7fa4527fc700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 0x7fa44006a720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa448000f60 tx=0x7fa448007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.364+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa444053470 con 0x7fa4540cd7c0 2023-11-28T11:17:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:41.570+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa4540d2af0 con 0x7fa4540cd7c0 2023-11-28T11:17:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.444+0000 7fa43b7fe700 1 -- 172.21.15.53:0/885022463 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa4440150a0 con 0x7fa4540cd7c0 2023-11-28T11:17:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.447+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 msgr2=0x7fa44006a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.447+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 0x7fa44006a720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa448000f60 tx=0x7fa448007d30 comp rx=0 tx=0).stop 2023-11-28T11:17:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.447+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 msgr2=0x7fa454143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.447+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa444014040 tx=0x7fa444004750 comp rx=0 tx=0).stop 2023-11-28T11:17:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 shutdown_connections 2023-11-28T11:17:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa440068270 0x7fa44006a720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa454161340 0x7fa4541e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540cd7c0 0x7fa454143200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 --2- 172.21.15.53:0/885022463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa45415c9b0 0x7fa4541e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 >> 172.21.15.53:0/885022463 conn(0x7fa454072600 msgr2=0x7fa45406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 shutdown_connections 2023-11-28T11:17:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.448+0000 7fa4593ff700 1 -- 172.21.15.53:0/885022463 wait complete. 2023-11-28T11:17:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'rbd' created 2023-11-28T11:17:42.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:17:42.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.682+0000 7faf74dff700 1 Processor -- start 2023-11-28T11:17:42.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.682+0000 7faf74dff700 1 -- start start 2023-11-28T11:17:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf74dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf74dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf7015dc30 0x7faf700b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf74dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7015e0a0 con 0x7faf700cd7c0 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf74dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7015ab30 con 0x7faf7015dc30 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf74dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf70160d70 con 0x7faf7015a760 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf67fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf7015dc30 0x7faf700b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.683+0000 7faf6e59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf6e59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:37924/0 (socket says 172.21.15.53:37924) 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf6e59c700 1 -- 172.21.15.53:0/227362848 learned_addr learned my addr 172.21.15.53:0/227362848 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf7015dc30 0x7faf700b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:36150/0 (socket says 172.21.15.53:36150) 2023-11-28T11:17:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf6ed9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faf700cd7c0 0x7faf700d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:58350/0 (socket says 172.21.15.53:58350) 2023-11-28T11:17:42.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 -- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 msgr2=0x7faf70161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 -- 172.21.15.53:0/227362848 >> v1:172.21.15.157:6789/0 conn(0x7faf700cd7c0 legacy=0x7faf700d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 -- 172.21.15.53:0/227362848 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700b5130 con 0x7faf7015dc30 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf6e59c700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.684+0000 7faf67fff700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7015dc30 0x7faf700b4bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf58009840 tx=0x7faf58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4306eef7ecacba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf6dd9b700 1 -- 172.21.15.53:0/227362848 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58011830 con 0x7faf7015dc30 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf6dd9b700 1 -- 172.21.15.53:0/227362848 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf58011990 con 0x7faf7015dc30 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf6dd9b700 1 -- 172.21.15.53:0/227362848 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58011d20 con 0x7faf7015dc30 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 -- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7015dc30 msgr2=0x7faf700b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7015dc30 0x7faf700b4bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf58009840 tx=0x7faf58009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 -- 172.21.15.53:0/227362848 shutdown_connections 2023-11-28T11:17:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015a760 0x7faf70161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 --2- 172.21.15.53:0/227362848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7015dc30 0x7faf700b4bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 -- 172.21.15.53:0/227362848 >> 172.21.15.53:0/227362848 conn(0x7faf70072620 msgr2=0x7faf700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 -- 172.21.15.53:0/227362848 shutdown_connections 2023-11-28T11:17:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.685+0000 7faf74dff700 1 -- 172.21.15.53:0/227362848 wait complete. 2023-11-28T11:17:42.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.688+0000 7faf74dff700 1 Processor -- start 2023-11-28T11:17:42.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.688+0000 7faf74dff700 1 -- start start 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf700cd7c0 0x7faf70155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015dc30 0x7faf700c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf70155120 con 0x7faf7015a760 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf70158170 con 0x7faf700cd7c0 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf74dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf70154e40 con 0x7faf7015dc30 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39466/0 (socket says 172.21.15.53:39466) 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 -- 172.21.15.53:0/992496398 learned_addr learned my addr 172.21.15.53:0/992496398 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf6e59c700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf700cd7c0 0x7faf70155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf6ed9d700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015dc30 0x7faf700c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 -- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015dc30 msgr2=0x7faf700c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015dc30 0x7faf700c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 -- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf700cd7c0 msgr2=0x7faf70155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf700cd7c0 0x7faf70155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.690+0000 7faf67fff700 1 -- 172.21.15.53:0/992496398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009550 con 0x7faf7015a760 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf67fff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf58009fd0 tx=0x7faf58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf580036a0 con 0x7faf7015a760 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5801e070 con 0x7faf7015a760 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf580094a0 con 0x7faf7015a760 2023-11-28T11:17:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58003a60 con 0x7faf7015a760 2023-11-28T11:17:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.691+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf700b5670 con 0x7faf7015a760 2023-11-28T11:17:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.692+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf700b5390 con 0x7faf7015a760 2023-11-28T11:17:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.694+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7faf5800f050 con 0x7faf7015a760 2023-11-28T11:17:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.696+0000 7faf66ffd700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 0x7faf5006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.696+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5199+0+0 (secure 0 0 0) 0x7faf58086c20 con 0x7faf7015a760 2023-11-28T11:17:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.696+0000 7faf6e59c700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 0x7faf5006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.697+0000 7faf6e59c700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 0x7faf5006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf60006d30 tx=0x7faf600092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:42.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.699+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf580538d0 con 0x7faf7015a760 2023-11-28T11:17:42.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:42.911+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faf70160720 con 0x7faf7015a760 2023-11-28T11:17:43.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.450+0000 7faf66ffd700 1 -- 172.21.15.53:0/992496398 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7faf58056ef0 con 0x7faf7015a760 2023-11-28T11:17:43.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 msgr2=0x7faf5006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 0x7faf5006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf60006d30 tx=0x7faf600092c0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 msgr2=0x7faf700b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf58009fd0 tx=0x7faf58004750 comp rx=0 tx=0).stop 2023-11-28T11:17:43.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 shutdown_connections 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf500682a0 0x7faf5006a750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7015dc30 0x7faf700c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf700cd7c0 0x7faf70155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 --2- 172.21.15.53:0/992496398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7015a760 0x7faf700b3200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.453+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 >> 172.21.15.53:0/992496398 conn(0x7faf70072620 msgr2=0x7faf70158430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.454+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 shutdown_connections 2023-11-28T11:17:43.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.455+0000 7faf74dff700 1 -- 172.21.15.53:0/992496398 wait complete. 2023-11-28T11:17:43.459 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:17:43.478 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:17:43.478 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:17:43.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:17:43.483 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:17:43.483 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:17:43.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:17:43.487 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:17:43.487 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:17:43.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:17:43.492 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:17:43.492 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:17:43.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:17:43.496 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:17:43.496 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:17:43.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:43.639 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:17:43.639 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:17:43.642 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-11-28T11:17:43.643 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-11-28T11:17:43.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.708+0000 7f384a7ff700 1 Processor -- start 2023-11-28T11:17:43.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.709+0000 7f384a7ff700 1 -- start start 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f384a7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f384415c9b0 0x7f38440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f384a7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3844161340 0x7f38440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f384a7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440d1950 con 0x7f38440d2320 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f384a7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440b5800 con 0x7f3844161340 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f384a7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440b5520 con 0x7f384415c9b0 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f38437fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3844161340 0x7f38440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f3843fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f384415c9b0 0x7f38440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f3848d9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f38440d2320 0x7f38440cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45208/0 (socket says 172.21.15.53:45208) 2023-11-28T11:17:43.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f3848d9c700 1 -- 172.21.15.53:0/3940974473 learned_addr learned my addr 172.21.15.53:0/3940974473 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f3842ffd700 1 -- 172.21.15.53:0/3940974473 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615997810 0 0) 0x7f38440d1950 con 0x7f38440d2320 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f38437fe700 1 -- 172.21.15.53:0/3940974473 >> v2:172.21.15.157:3300/0 conn(0x7f384415c9b0 msgr2=0x7f38440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f38437fe700 1 --2- 172.21.15.53:0/3940974473 >> v2:172.21.15.157:3300/0 conn(0x7f384415c9b0 0x7f38440b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.710+0000 7f38437fe700 1 -- 172.21.15.53:0/3940974473 >> v1:172.21.15.53:6789/0 conn(0x7f38440d2320 legacy=0x7f38440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.711+0000 7f38437fe700 1 -- 172.21.15.53:0/3940974473 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440d1300 con 0x7f3844161340 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.711+0000 7f38437fe700 1 --2- 172.21.15.53:0/3940974473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440d1de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3844067460 tx=0x7f3838009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375c7a6b536121a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.711+0000 7f3842ffd700 1 -- 172.21.15.53:0/3940974473 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838011830 con 0x7f3844161340 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.711+0000 7f3842ffd700 1 -- 172.21.15.53:0/3940974473 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3838011990 con 0x7f3844161340 2023-11-28T11:17:43.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.711+0000 7f3842ffd700 1 -- 172.21.15.53:0/3940974473 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838011d20 con 0x7f3844161340 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 -- 172.21.15.53:0/3940974473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 msgr2=0x7f38440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3940974473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440d1de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3844067460 tx=0x7f3838009b20 comp rx=0 tx=0).stop 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 -- 172.21.15.53:0/3940974473 shutdown_connections 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3940974473 >> v2:172.21.15.157:3300/0 conn(0x7f384415c9b0 0x7f38440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3940974473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440d1de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 -- 172.21.15.53:0/3940974473 >> 172.21.15.53:0/3940974473 conn(0x7f3844072600 msgr2=0x7f38440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 -- 172.21.15.53:0/3940974473 shutdown_connections 2023-11-28T11:17:43.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.712+0000 7f384a7ff700 1 -- 172.21.15.53:0/3940974473 wait complete. 2023-11-28T11:17:43.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.713+0000 7f384a7ff700 1 Processor -- start 2023-11-28T11:17:43.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.714+0000 7f384a7ff700 1 -- start start 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38440d2320 0x7f384414c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f384415c9b0 0x7f38440ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384414cae0 con 0x7f3844161340 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440cba50 con 0x7f38440d2320 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f384a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440cbd30 con 0x7f384415c9b0 2023-11-28T11:17:43.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3843fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38440d2320 0x7f384414c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39516/0 (socket says 172.21.15.53:39516) 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 -- 172.21.15.53:0/3933438138 learned_addr learned my addr 172.21.15.53:0/3933438138 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f38437fe700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f384415c9b0 0x7f38440ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 -- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f384415c9b0 msgr2=0x7f38440ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f384415c9b0 0x7f38440ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 -- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38440d2320 msgr2=0x7f384414c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38440d2320 0x7f384414c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.715+0000 7f3848d9c700 1 -- 172.21.15.53:0/3933438138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380094a0 con 0x7f3844161340 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f3848d9c700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f383c00cc10 tx=0x7f383c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00aaa0 con 0x7f3844161340 2023-11-28T11:17:43.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00ac20 con 0x7f3844161340 2023-11-28T11:17:43.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38440b5130 con 0x7f3844161340 2023-11-28T11:17:43.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0070d0 con 0x7f3844161340 2023-11-28T11:17:43.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.716+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440d15e0 con 0x7f3844161340 2023-11-28T11:17:43.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.717+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38440b52c0 con 0x7f3844161340 2023-11-28T11:17:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.718+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f383c01a030 con 0x7f3844161340 2023-11-28T11:17:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.719+0000 7f3840ff9700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 0x7f383006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.719+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f383c085560 con 0x7f3844161340 2023-11-28T11:17:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.719+0000 7f3843fff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 0x7f383006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.720+0000 7f3843fff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 0x7f383006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38440caf40 tx=0x7f3834009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.723+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f383c052070 con 0x7f3844161340 2023-11-28T11:17:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.795+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f383c010070 con 0x7f3844161340 2023-11-28T11:17:43.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.935+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38440745c0 con 0x7f3844161340 2023-11-28T11:17:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.936+0000 7f3840ff9700 1 -- 172.21.15.53:0/3933438138 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14277 (secure 0 0 0) 0x7f383c055aa0 con 0x7f3844161340 2023-11-28T11:17:43.937 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:43.938 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:43.443489+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 msgr2=0x7f383006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 0x7f383006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38440caf40 tx=0x7f3834009210 comp rx=0 tx=0).stop 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 msgr2=0x7f38440cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f383c00cc10 tx=0x7f383c00cf20 comp rx=0 tx=0).stop 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 shutdown_connections 2023-11-28T11:17:43.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38300682a0 0x7f383006a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844161340 0x7f38440cafb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38440d2320 0x7f384414c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 --2- 172.21.15.53:0/3933438138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f384415c9b0 0x7f38440ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 >> 172.21.15.53:0/3933438138 conn(0x7f3844072600 msgr2=0x7f384415a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 shutdown_connections 2023-11-28T11:17:43.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:43.939+0000 7f384a7ff700 1 -- 172.21.15.53:0/3933438138 wait complete. 2023-11-28T11:17:43.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:17:43.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:17:44.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.164+0000 7f461e5ff700 1 Processor -- start 2023-11-28T11:17:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.165+0000 7f461e5ff700 1 -- start start 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f461e5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f461815c9b0 0x7f46180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f461e5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180d1950 con 0x7f4618161340 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f461e5ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180d24f0 con 0x7f46180cd7c0 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f461e5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180b5800 con 0x7f461815c9b0 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f4617fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f461815c9b0 0x7f46180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f46177fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4618161340 0x7f46180d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45210/0 (socket says 172.21.15.53:45210) 2023-11-28T11:17:44.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f46177fe700 1 -- 172.21.15.53:0/3105428593 learned_addr learned my addr 172.21.15.53:0/3105428593 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.166+0000 7f4617fff700 1 -- 172.21.15.53:0/3105428593 >> v1:172.21.15.157:6789/0 conn(0x7f46180cd7c0 legacy=0x7f46180b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4617fff700 1 -- 172.21.15.53:0/3105428593 >> v1:172.21.15.53:6789/0 conn(0x7f4618161340 legacy=0x7f46180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4617fff700 1 -- 172.21.15.53:0/3105428593 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180b5520 con 0x7f461815c9b0 2023-11-28T11:17:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4616ffd700 1 -- 172.21.15.53:0/3105428593 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588596173 0 0) 0x7f46180d1950 con 0x7f4618161340 2023-11-28T11:17:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4617fff700 1 --2- 172.21.15.53:0/3105428593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f461815c9b0 0x7f46180b4bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4608009840 tx=0x7f4608009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0c5b2d823e17e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4616ffd700 1 -- 172.21.15.53:0/3105428593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608004230 con 0x7f461815c9b0 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4616ffd700 1 -- 172.21.15.53:0/3105428593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608004390 con 0x7f461815c9b0 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.167+0000 7f4616ffd700 1 -- 172.21.15.53:0/3105428593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080036a0 con 0x7f461815c9b0 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 -- 172.21.15.53:0/3105428593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f461815c9b0 msgr2=0x7f46180b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 --2- 172.21.15.53:0/3105428593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f461815c9b0 0x7f46180b4bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4608009840 tx=0x7f4608009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 -- 172.21.15.53:0/3105428593 shutdown_connections 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 --2- 172.21.15.53:0/3105428593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f461815c9b0 0x7f46180b4bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 -- 172.21.15.53:0/3105428593 >> 172.21.15.53:0/3105428593 conn(0x7f4618072600 msgr2=0x7f46180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 -- 172.21.15.53:0/3105428593 shutdown_connections 2023-11-28T11:17:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.168+0000 7f461e5ff700 1 -- 172.21.15.53:0/3105428593 wait complete. 2023-11-28T11:17:44.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.170+0000 7f461e5ff700 1 Processor -- start 2023-11-28T11:17:44.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.170+0000 7f461e5ff700 1 -- start start 2023-11-28T11:17:44.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f461815c9b0 0x7f46180ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 0x7f46180cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180cd5c0 con 0x7f4618161340 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461815e650 con 0x7f461815c9b0 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461e5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461815e930 con 0x7f46180cd7c0 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461cb9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 0x7f46180cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:37964/0 (socket says 172.21.15.53:37964) 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 -- 172.21.15.53:0/597422343 learned_addr learned my addr 172.21.15.53:0/597422343 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f46177fe700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f461815c9b0 0x7f46180ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 -- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f461815c9b0 msgr2=0x7f46180ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f461815c9b0 0x7f46180ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 -- 172.21.15.53:0/597422343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 msgr2=0x7f46180cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 0x7f46180cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f4617fff700 1 -- 172.21.15.53:0/597422343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080094a0 con 0x7f46180cd7c0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.171+0000 7f461cb9c700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 0x7f46180cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f4617fff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46180675d0 tx=0x7f46080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608011630 con 0x7f46180cd7c0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608011790 con 0x7f46180cd7c0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608011b40 con 0x7f46180cd7c0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46180d15e0 con 0x7f46180cd7c0 2023-11-28T11:17:44.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.172+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46180d1300 con 0x7f46180cd7c0 2023-11-28T11:17:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.173+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46181f37b0 con 0x7f46180cd7c0 2023-11-28T11:17:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.174+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4608017020 con 0x7f46180cd7c0 2023-11-28T11:17:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.175+0000 7f4614ff9700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 0x7f460406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.175+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f46080871b0 con 0x7f46180cd7c0 2023-11-28T11:17:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.175+0000 7f46177fe700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 0x7f460406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.176+0000 7f46177fe700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 0x7f460406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f460c000f60 tx=0x7f460c007c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.180+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4608053c70 con 0x7f46180cd7c0 2023-11-28T11:17:44.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:17:44.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:17:44.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.384+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46180745c0 con 0x7f46180cd7c0 2023-11-28T11:17:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.435+0000 7f4614ff9700 1 -- 172.21.15.53:0/597422343 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4608057290 con 0x7f46180cd7c0 2023-11-28T11:17:44.436 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:17:44.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 msgr2=0x7f460406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 0x7f460406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f460c000f60 tx=0x7f460c007c40 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 msgr2=0x7f46181f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46180675d0 tx=0x7f46080047c0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 shutdown_connections 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f46040682f0 0x7f460406a7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46180cd7c0 0x7f46181f52d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4618161340 0x7f46180cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 --2- 172.21.15.53:0/597422343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f461815c9b0 0x7f46180ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 >> 172.21.15.53:0/597422343 conn(0x7f4618072600 msgr2=0x7f46180b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 shutdown_connections 2023-11-28T11:17:44.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.437+0000 7f461e5ff700 1 -- 172.21.15.53:0/597422343 wait complete. 2023-11-28T11:17:44.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:17:44.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.704+0000 7ff45a9ff700 1 Processor -- start 2023-11-28T11:17:44.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.704+0000 7ff45a9ff700 1 -- start start 2023-11-28T11:17:44.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff45a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff4540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff45a9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4540cd7c0 0x7ff4540cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff45a9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4540ce160 con 0x7ff454161340 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff45a9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540b5800 con 0x7ff4540cd7c0 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff45a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540d1950 con 0x7ff45415c9b0 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff4537fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff454161340 0x7ff4540d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45218/0 (socket says 172.21.15.53:45218) 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.705+0000 7ff4537fe700 1 -- 172.21.15.53:0/3260987381 learned_addr learned my addr 172.21.15.53:0/3260987381 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff458f9c700 1 --2- 172.21.15.53:0/3260987381 >> v2:172.21.15.157:3300/0 conn(0x7ff4540cd7c0 0x7ff4540cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff453fff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff4540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673749874 0 0) 0x7ff4540ce160 con 0x7ff454161340 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff43c0035b0 con 0x7ff454161340 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 789983214 0 0) 0x7ff43c0035b0 con 0x7ff454161340 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 msgr2=0x7ff4540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff4540b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 >> v2:172.21.15.157:3300/0 conn(0x7ff4540cd7c0 msgr2=0x7ff4540cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 --2- 172.21.15.53:0/3260987381 >> v2:172.21.15.157:3300/0 conn(0x7ff4540cd7c0 0x7ff4540cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540d24f0 con 0x7ff454161340 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff458f9c700 1 --2- 172.21.15.53:0/3260987381 >> v2:172.21.15.157:3300/0 conn(0x7ff4540cd7c0 0x7ff4540cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7ff448001670 con 0x7ff454161340 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 >> v1:172.21.15.53:6789/0 conn(0x7ff454161340 legacy=0x7ff4540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff43c003a00 0x7ff43c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c004430 0x7ff43c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff43c005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540b5800 con 0x7ff43c004430 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540d1950 con 0x7ff43c003a00 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.706+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c00ba10 con 0x7ff45415c9b0 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff43c003a00 0x7ff43c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c004430 0x7ff43c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff458f9c700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff43c005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 -- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 msgr2=0x7ff43c005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415c9b0 0x7ff43c005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 -- 172.21.15.53:0/3260987381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c004430 msgr2=0x7ff43c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c004430 0x7ff43c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 -- 172.21.15.53:0/3260987381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0035b0 con 0x7ff43c003a00 2023-11-28T11:17:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c004430 0x7ff43c0048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.707+0000 7ff453fff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff43c003a00 0x7ff43c003e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff444009b90 tx=0x7ff43c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb21fa107aee260 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400e6c0 con 0x7ff43c003a00 2023-11-28T11:17:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400e820 con 0x7ff43c003a00 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff452ffd700 1 -- 172.21.15.53:0/3260987381 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400eb90 con 0x7ff43c003a00 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff43c003a00 msgr2=0x7ff43c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3260987381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff43c003a00 0x7ff43c003e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff444009b90 tx=0x7ff43c005880 comp rx=0 tx=0).stop 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff458f9c700 1 -- 172.21.15.53:0/3260987381 reap_dead start 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3260987381 shutdown_connections 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3260987381 >> 172.21.15.53:0/3260987381 conn(0x7ff454072600 msgr2=0x7ff4540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3260987381 shutdown_connections 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.708+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3260987381 wait complete. 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.710+0000 7ff45a9ff700 1 Processor -- start 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.710+0000 7ff45a9ff700 1 -- start start 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 0x7ff4540cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cb190 0x7ff4540ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541f2700 con 0x7ff43c003a00 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541f3220 con 0x7ff4541f36b0 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff45a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541f28f0 con 0x7ff4540cb190 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff453fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff453fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39542/0 (socket says 172.21.15.53:39542) 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff453fff700 1 -- 172.21.15.53:0/3017655452 learned_addr learned my addr 172.21.15.53:0/3017655452 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 0x7ff4540cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.711+0000 7ff458f9c700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cb190 0x7ff4540ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 -- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cb190 msgr2=0x7ff4540ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cb190 0x7ff4540ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 -- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 msgr2=0x7ff4540cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 0x7ff4540cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 -- 172.21.15.53:0/3017655452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540d1300 con 0x7ff43c003a00 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 0x7ff4540cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff453fff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0057f0 tx=0x7ff4440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400e6c0 con 0x7ff43c003a00 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4440038a0 con 0x7ff43c003a00 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540b5520 con 0x7ff43c003a00 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.712+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444003c30 con 0x7ff43c003a00 2023-11-28T11:17:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.713+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4540d15e0 con 0x7ff43c003a00 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.714+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4540745c0 con 0x7ff43c003a00 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.715+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff444019030 con 0x7ff43c003a00 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.716+0000 7ff450ff9700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 0x7ff44006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.716+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff444086f50 con 0x7ff43c003a00 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.716+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 0x7ff44006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.716+0000 7ff4537fe700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 0x7ff44006a7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4541f3a80 tx=0x7ff448007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:44.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.721+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff4440b85e0 con 0x7ff43c003a00 2023-11-28T11:17:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.922+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4540d08a0 con 0x7ff43c003a00 2023-11-28T11:17:44.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.923+0000 7ff450ff9700 1 -- 172.21.15.53:0/3017655452 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff444057530 con 0x7ff43c003a00 2023-11-28T11:17:44.924 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:17:44.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 msgr2=0x7ff44006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 0x7ff44006a7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4541f3a80 tx=0x7ff448007980 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 msgr2=0x7ff4541579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0057f0 tx=0x7ff4440047c0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 shutdown_connections 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4400682f0 0x7ff44006a7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cb190 0x7ff4540ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c003a00 0x7ff4541579f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 --2- 172.21.15.53:0/3017655452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4541f36b0 0x7ff4540cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 >> 172.21.15.53:0/3017655452 conn(0x7ff454072600 msgr2=0x7ff45415bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 shutdown_connections 2023-11-28T11:17:44.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:44.925+0000 7ff45a9ff700 1 -- 172.21.15.53:0/3017655452 wait complete. 2023-11-28T11:17:44.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:17:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.143+0000 7f0013dff700 1 Processor -- start 2023-11-28T11:17:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.143+0000 7f0013dff700 1 -- start start 2023-11-28T11:17:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.144+0000 7f0013dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f000c15c9b0 0x7f000c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.144+0000 7f0013dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0013dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c0d1950 con 0x7f000c0d2320 2023-11-28T11:17:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0013dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0b5800 con 0x7f000c15c9b0 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0013dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b5520 con 0x7f000c161340 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0011b9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f000c15c9b0 0x7f000c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f001239c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f000c0d2320 0x7f000c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45220/0 (socket says 172.21.15.53:45220) 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f001239c700 1 -- 172.21.15.53:0/2352125475 learned_addr learned my addr 172.21.15.53:0/2352125475 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f001139a700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0010b99700 1 -- 172.21.15.53:0/2352125475 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376053206 0 0) 0x7f000c0d1950 con 0x7f000c0d2320 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0010b99700 1 -- 172.21.15.53:0/2352125475 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff80035b0 con 0x7f000c0d2320 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0011b9b700 1 -- 172.21.15.53:0/2352125475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 msgr2=0x7f000c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0011b9b700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0011b9b700 1 -- 172.21.15.53:0/2352125475 >> v1:172.21.15.53:6789/0 conn(0x7f000c0d2320 legacy=0x7f000c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f0011b9b700 1 -- 172.21.15.53:0/2352125475 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0d1300 con 0x7f000c15c9b0 2023-11-28T11:17:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.145+0000 7f001139a700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0011b9b700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c15c9b0 0x7f000c0b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0008009840 tx=0x7f0008009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3939a9928c71b9d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0010b99700 1 -- 172.21.15.53:0/2352125475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008011830 con 0x7f000c15c9b0 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0010b99700 1 -- 172.21.15.53:0/2352125475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008011990 con 0x7f000c15c9b0 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0010b99700 1 -- 172.21.15.53:0/2352125475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008011d20 con 0x7f000c15c9b0 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0013dff700 1 -- 172.21.15.53:0/2352125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c15c9b0 msgr2=0x7f000c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.146+0000 7f0013dff700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c15c9b0 0x7f000c0b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0008009840 tx=0x7f0008009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 -- 172.21.15.53:0/2352125475 shutdown_connections 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c15c9b0 0x7f000c0b4bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 --2- 172.21.15.53:0/2352125475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 -- 172.21.15.53:0/2352125475 >> 172.21.15.53:0/2352125475 conn(0x7f000c072600 msgr2=0x7f000c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 -- 172.21.15.53:0/2352125475 shutdown_connections 2023-11-28T11:17:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.147+0000 7f0013dff700 1 -- 172.21.15.53:0/2352125475 wait complete. 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.148+0000 7f0013dff700 1 Processor -- start 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.149+0000 7f0013dff700 1 -- start start 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c15c9b0 0x7f000c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153e30 con 0x7f000c0d2320 2023-11-28T11:17:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153220 con 0x7f000c15c9b0 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0013dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153500 con 0x7f000c161340 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39562/0 (socket says 172.21.15.53:39562) 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 -- 172.21.15.53:0/99730743 learned_addr learned my addr 172.21.15.53:0/99730743 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 -- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 msgr2=0x7f000c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 -- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c15c9b0 msgr2=0x7f000c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c15c9b0 0x7f000c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 -- 172.21.15.53:0/99730743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0094a0 con 0x7f000c0d2320 2023-11-28T11:17:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.150+0000 7f0011b9b700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0008009fd0 tx=0x7f0008004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.151+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080036a0 con 0x7f000c0d2320 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.151+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000801e070 con 0x7f000c0d2320 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.151+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008003a40 con 0x7f000c0d2320 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.151+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00080094a0 con 0x7f000c0d2320 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.151+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c0b5130 con 0x7f000c0d2320 2023-11-28T11:17:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.152+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c0d15e0 con 0x7f000c0d2320 2023-11-28T11:17:45.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.153+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f000800f050 con 0x7f000c0d2320 2023-11-28T11:17:45.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.154+0000 7f00027fc700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 0x7efff006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.154+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0008086990 con 0x7f000c0d2320 2023-11-28T11:17:45.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.154+0000 7f001139a700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 0x7efff006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.155+0000 7f001139a700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 0x7efff006a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efffc006f10 tx=0x7efffc006e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.158+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0008053450 con 0x7f000c0d2320 2023-11-28T11:17:45.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.403+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f000c0b52c0 con 0x7f000c0d2320 2023-11-28T11:17:45.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.404+0000 7f00027fc700 1 -- 172.21.15.53:0/99730743 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3532 (secure 0 0 0) 0x7f0008015090 con 0x7f000c0d2320 2023-11-28T11:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":54179,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 msgr2=0x7efff006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 0x7efff006a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efffc006f10 tx=0x7efffc006e60 comp rx=0 tx=0).stop 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 msgr2=0x7f000c1542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0008009fd0 tx=0x7f0008004750 comp rx=0 tx=0).stop 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 shutdown_connections 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efff00682f0 0x7efff006a7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c161340 0x7f000c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000c0d2320 0x7f000c1542c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 --2- 172.21.15.53:0/99730743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c15c9b0 0x7f000c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 >> 172.21.15.53:0/99730743 conn(0x7f000c072600 msgr2=0x7f000c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 shutdown_connections 2023-11-28T11:17:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.406+0000 7f0013dff700 1 -- 172.21.15.53:0/99730743 wait complete. 2023-11-28T11:17:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 3 2023-11-28T11:17:45.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:45.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.627+0000 7f195d9ff700 1 Processor -- start 2023-11-28T11:17:45.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.627+0000 7f195d9ff700 1 -- start start 2023-11-28T11:17:45.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.628+0000 7f195d9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19580d1330 0x7f19580b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.628+0000 7f195d9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 0x7f19580b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.628+0000 7f195d9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19580b0230 con 0x7f19580d01b0 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.628+0000 7f195d9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195815cec0 con 0x7f19580d1330 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.628+0000 7f195d9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19580b4570 con 0x7f19580b20b0 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19580d1330 0x7f19580b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19580d1330 0x7f19580b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39564/0 (socket says 172.21.15.53:39564) 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 -- 172.21.15.53:0/4168681759 learned_addr learned my addr 172.21.15.53:0/4168681759 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19577fe700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 0x7f19580b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 -- 172.21.15.53:0/4168681759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 msgr2=0x7f19580b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 0x7f19580b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 -- 172.21.15.53:0/4168681759 >> v1:172.21.15.157:6789/0 conn(0x7f19580d01b0 legacy=0x7f19580d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 -- 172.21.15.53:0/4168681759 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580b4850 con 0x7f19580d1330 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19577fe700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 0x7f19580b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f19567fc700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580d1330 0x7f19580b06c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098f0 tx=0x7f194c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc6d8a15c55a361 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.629+0000 7f1955ffb700 1 -- 172.21.15.53:0/4168681759 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00b8a0 con 0x7f19580d1330 2023-11-28T11:17:45.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f1955ffb700 1 -- 172.21.15.53:0/4168681759 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00ba00 con 0x7f19580d1330 2023-11-28T11:17:45.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f1955ffb700 1 -- 172.21.15.53:0/4168681759 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00bd90 con 0x7f19580d1330 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 -- 172.21.15.53:0/4168681759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580d1330 msgr2=0x7f19580b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580d1330 0x7f19580b06c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098f0 tx=0x7f194c009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 -- 172.21.15.53:0/4168681759 shutdown_connections 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580b20b0 0x7f19580b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 --2- 172.21.15.53:0/4168681759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580d1330 0x7f19580b06c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 -- 172.21.15.53:0/4168681759 >> 172.21.15.53:0/4168681759 conn(0x7f195809c3f0 msgr2=0x7f19580d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 -- 172.21.15.53:0/4168681759 shutdown_connections 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.630+0000 7f195d9ff700 1 -- 172.21.15.53:0/4168681759 wait complete. 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.632+0000 7f195d9ff700 1 Processor -- start 2023-11-28T11:17:45.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.632+0000 7f195d9ff700 1 -- start start 2023-11-28T11:17:45.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19580d01b0 0x7f195815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580d1330 0x7f19580b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580cc190 con 0x7f19580b20b0 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19581f32b0 con 0x7f19580d01b0 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f195d9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19580cc450 con 0x7f19580d1330 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f1956ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f1956ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39568/0 (socket says 172.21.15.53:39568) 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f1956ffd700 1 -- 172.21.15.53:0/3420748786 learned_addr learned my addr 172.21.15.53:0/3420748786 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f19567fc700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19580d01b0 0x7f195815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.633+0000 7f19577fe700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580d1330 0x7f19580b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 -- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580d1330 msgr2=0x7f19580b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580d1330 0x7f19580b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 -- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19580d01b0 msgr2=0x7f195815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19580d01b0 0x7f195815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 -- 172.21.15.53:0/3420748786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009550 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f1956ffd700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f194800c4e0 tx=0x7f194800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480052d0 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1948005430 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19580afec0 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480038c0 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.634+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19580afbe0 con 0x7f19580b20b0 2023-11-28T11:17:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.635+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19580b2600 con 0x7f19580b20b0 2023-11-28T11:17:45.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.636+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1948019030 con 0x7f19580b20b0 2023-11-28T11:17:45.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.637+0000 7f193ffff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 0x7f194406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:45.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.637+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1948084d60 con 0x7f19580b20b0 2023-11-28T11:17:45.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.638+0000 7f19567fc700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 0x7f194406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:45.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.638+0000 7f19567fc700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 0x7f194406a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19580d1720 tx=0x7f194c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:45.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.641+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1948051820 con 0x7f19580b20b0 2023-11-28T11:17:45.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.845+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1958068690 con 0x7f19580b20b0 2023-11-28T11:17:45.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.846+0000 7f193ffff700 1 -- 172.21.15.53:0/3420748786 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14264 (secure 0 0 0) 0x7f1948054e40 con 0x7f19580b20b0 2023-11-28T11:17:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:45.847 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:44.449285+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 msgr2=0x7f194406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 0x7f194406a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19580d1720 tx=0x7f194c008030 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 msgr2=0x7f19580ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f194800c4e0 tx=0x7f194800c7f0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 shutdown_connections 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19440682f0 0x7f194406a7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19580d1330 0x7f19580b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19580b20b0 0x7f19580ccc50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 --2- 172.21.15.53:0/3420748786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19580d01b0 0x7f195815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 >> 172.21.15.53:0/3420748786 conn(0x7f195809c3f0 msgr2=0x7f19580cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 shutdown_connections 2023-11-28T11:17:45.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:45.848+0000 7f195d9ff700 1 -- 172.21.15.53:0/3420748786 wait complete. 2023-11-28T11:17:45.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:17:45.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:17:46.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.072+0000 7f14d83ff700 1 Processor -- start 2023-11-28T11:17:46.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.072+0000 7f14d83ff700 1 -- start start 2023-11-28T11:17:46.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d83ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 0x7f14d00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d83ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14d00b20b0 0x7f14d00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d83ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d00b0230 con 0x7f14d00d01b0 2023-11-28T11:17:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d83ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d015cec0 con 0x7f14d00b20b0 2023-11-28T11:17:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d83ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b4570 con 0x7f14d00d1330 2023-11-28T11:17:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d699c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14d00b20b0 0x7f14d00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d699c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14d00b20b0 0x7f14d00b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39572/0 (socket says 172.21.15.53:39572) 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.073+0000 7f14d699c700 1 -- 172.21.15.53:0/3327217204 learned_addr learned my addr 172.21.15.53:0/3327217204 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d599a700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 0x7f14d00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d699c700 1 -- 172.21.15.53:0/3327217204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 msgr2=0x7f14d00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d699c700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 0x7f14d00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d699c700 1 -- 172.21.15.53:0/3327217204 >> v1:172.21.15.157:6789/0 conn(0x7f14d00d01b0 legacy=0x7f14d00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d699c700 1 -- 172.21.15.53:0/3327217204 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00b4850 con 0x7f14d00b20b0 2023-11-28T11:17:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d599a700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 0x7f14d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d699c700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00b20b0 0x7f14d00b0c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009e50 tx=0x7f14c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90913b471b52d75b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d5199700 1 -- 172.21.15.53:0/3327217204 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800e070 con 0x7f14d00b20b0 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d5199700 1 -- 172.21.15.53:0/3327217204 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8003a20 con 0x7f14d00b20b0 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.074+0000 7f14d5199700 1 -- 172.21.15.53:0/3327217204 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8003dd0 con 0x7f14d00b20b0 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 -- 172.21.15.53:0/3327217204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00b20b0 msgr2=0x7f14d00b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00b20b0 0x7f14d00b0c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009e50 tx=0x7f14c800b330 comp rx=0 tx=0).stop 2023-11-28T11:17:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 -- 172.21.15.53:0/3327217204 shutdown_connections 2023-11-28T11:17:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00b20b0 0x7f14d00b0c00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3327217204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d1330 0x7f14d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 -- 172.21.15.53:0/3327217204 >> 172.21.15.53:0/3327217204 conn(0x7f14d009c3f0 msgr2=0x7f14d00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 -- 172.21.15.53:0/3327217204 shutdown_connections 2023-11-28T11:17:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.075+0000 7f14d83ff700 1 -- 172.21.15.53:0/3327217204 wait complete. 2023-11-28T11:17:46.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.077+0000 7f14d83ff700 1 Processor -- start 2023-11-28T11:17:46.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.077+0000 7f14d83ff700 1 -- start start 2023-11-28T11:17:46.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 0x7f14d009f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d01b0 0x7f14d009cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d009d2a0 con 0x7f14d00d1330 2023-11-28T11:17:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d009e0a0 con 0x7f14d00b20b0 2023-11-28T11:17:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d83ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d009d490 con 0x7f14d00d01b0 2023-11-28T11:17:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d699c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d699c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39586/0 (socket says 172.21.15.53:39586) 2023-11-28T11:17:46.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d699c700 1 -- 172.21.15.53:0/3547889992 learned_addr learned my addr 172.21.15.53:0/3547889992 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:46.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d599a700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d01b0 0x7f14d009cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.078+0000 7f14d619b700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 0x7f14d009f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 -- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d01b0 msgr2=0x7f14d009cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d01b0 0x7f14d009cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 -- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 msgr2=0x7f14d009f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 0x7f14d009f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 -- 172.21.15.53:0/3547889992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c8009b30 con 0x7f14d00d1330 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d619b700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 0x7f14d009f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:46.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d699c700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b5c40 tx=0x7f14c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80092e0 con 0x7f14d00d1330 2023-11-28T11:17:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8004240 con 0x7f14d00d1330 2023-11-28T11:17:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00afec0 con 0x7f14d00d1330 2023-11-28T11:17:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004610 con 0x7f14d00d1330 2023-11-28T11:17:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.079+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d00afbe0 con 0x7f14d00d1330 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.080+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d00b2600 con 0x7f14d00d1330 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.081+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f14c8004770 con 0x7f14d00d1330 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.082+0000 7f14c6ffd700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 0x7f14b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.082+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f14c8086da0 con 0x7f14d00d1330 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.082+0000 7f14d619b700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 0x7f14b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.083+0000 7f14d619b700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 0x7f14b406a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003fc0 tx=0x7f14cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.086+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f14c8053860 con 0x7f14d00d1330 2023-11-28T11:17:46.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.286+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14d015d000 con 0x7f14d00d1330 2023-11-28T11:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.287+0000 7f14c6ffd700 1 -- 172.21.15.53:0/3547889992 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14c8020020 con 0x7f14d00d1330 2023-11-28T11:17:46.288 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:17:46.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 msgr2=0x7f14b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 0x7f14b406a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003fc0 tx=0x7f14cc009040 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 msgr2=0x7f14d014f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b5c40 tx=0x7f14c8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 shutdown_connections 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f14b40682f0 0x7f14b406a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d00d1330 0x7f14d014f840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d00b20b0 0x7f14d009f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 --2- 172.21.15.53:0/3547889992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d00d01b0 0x7f14d009cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 >> 172.21.15.53:0/3547889992 conn(0x7f14d009c3f0 msgr2=0x7f14d015c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 shutdown_connections 2023-11-28T11:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.289+0000 7f14d83ff700 1 -- 172.21.15.53:0/3547889992 wait complete. 2023-11-28T11:17:46.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:17:46.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.512+0000 7f1848dff700 1 Processor -- start 2023-11-28T11:17:46.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.512+0000 7f1848dff700 1 -- start start 2023-11-28T11:17:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f1848dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18440d1330 0x7f18440b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f1848dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440b0230 con 0x7f18440d01b0 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f1848dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440b4570 con 0x7f18440b20b0 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f1848dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184415cec0 con 0x7f18440d1330 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f184259c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f18440d01b0 0x7f18440d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45228/0 (socket says 172.21.15.53:45228) 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f183bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18440d1330 0x7f18440b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.513+0000 7f184259c700 1 -- 172.21.15.53:0/4015532915 learned_addr learned my addr 172.21.15.53:0/4015532915 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f183bfff700 1 -- 172.21.15.53:0/4015532915 >> v1:172.21.15.157:6789/0 conn(0x7f18440b20b0 legacy=0x7f18440b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f183bfff700 1 -- 172.21.15.53:0/4015532915 >> v1:172.21.15.53:6789/0 conn(0x7f18440d01b0 legacy=0x7f18440d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f183bfff700 1 -- 172.21.15.53:0/4015532915 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440b4850 con 0x7f18440d1330 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f183bfff700 1 --2- 172.21.15.53:0/4015532915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d1330 0x7f18440b06c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f182c009840 tx=0x7f182c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b404d0ce81c75c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f1841d9b700 1 -- 172.21.15.53:0/4015532915 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c011830 con 0x7f18440d1330 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f1841d9b700 1 -- 172.21.15.53:0/4015532915 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c011990 con 0x7f18440d1330 2023-11-28T11:17:46.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.514+0000 7f1841d9b700 1 -- 172.21.15.53:0/4015532915 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c011d20 con 0x7f18440d1330 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 -- 172.21.15.53:0/4015532915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d1330 msgr2=0x7f18440b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 --2- 172.21.15.53:0/4015532915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d1330 0x7f18440b06c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f182c009840 tx=0x7f182c009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 -- 172.21.15.53:0/4015532915 shutdown_connections 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 --2- 172.21.15.53:0/4015532915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d1330 0x7f18440b06c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 -- 172.21.15.53:0/4015532915 >> 172.21.15.53:0/4015532915 conn(0x7f184409c3f0 msgr2=0x7f18440d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 -- 172.21.15.53:0/4015532915 shutdown_connections 2023-11-28T11:17:46.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.515+0000 7f1848dff700 1 -- 172.21.15.53:0/4015532915 wait complete. 2023-11-28T11:17:46.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.517+0000 7f1848dff700 1 Processor -- start 2023-11-28T11:17:46.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.517+0000 7f1848dff700 1 -- start start 2023-11-28T11:17:46.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 0x7f18441f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18440d1330 0x7f18440a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184409e930 con 0x7f18440d01b0 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184409ec10 con 0x7f18440b20b0 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1848dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184409eef0 con 0x7f18440d1330 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39616/0 (socket says 172.21.15.53:39616) 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 -- 172.21.15.53:0/1576465003 learned_addr learned my addr 172.21.15.53:0/1576465003 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f184259c700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 0x7f18441f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f1842d9d700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18440d1330 0x7f18440a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 -- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18440d1330 msgr2=0x7f18440a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18440d1330 0x7f18440a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 -- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 msgr2=0x7f18441f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 0x7f18441f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f183bfff700 1 -- 172.21.15.53:0/1576465003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0094a0 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.518+0000 7f184259c700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 0x7f18441f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f183bfff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f182c000c00 tx=0x7f182c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0038a0 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c01e070 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c003c60 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440afec0 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.519+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440afbe0 con 0x7f18440d01b0 2023-11-28T11:17:46.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.520+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18440b2600 con 0x7f18440d01b0 2023-11-28T11:17:46.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.521+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f182c00f050 con 0x7f18440d01b0 2023-11-28T11:17:46.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.522+0000 7f183affd700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 0x7f182406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.522+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f182c086bb0 con 0x7f18440d01b0 2023-11-28T11:17:46.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.522+0000 7f184259c700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 0x7f182406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.523+0000 7f184259c700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 0x7f182406a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1834007fd0 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.526+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f182c053670 con 0x7f18440d01b0 2023-11-28T11:17:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.730+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1844068690 con 0x7f18440d01b0 2023-11-28T11:17:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.730+0000 7f183affd700 1 -- 172.21.15.53:0/1576465003 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f182c0150a0 con 0x7f18440d01b0 2023-11-28T11:17:46.732 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 msgr2=0x7f182406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 0x7f182406a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1834007fd0 tx=0x7f1834009040 comp rx=0 tx=0).stop 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 msgr2=0x7f1844147e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f182c000c00 tx=0x7f182c004750 comp rx=0 tx=0).stop 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 shutdown_connections 2023-11-28T11:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f18240682f0 0x7f182406a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18440d1330 0x7f18440a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440b20b0 0x7f18441f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 --2- 172.21.15.53:0/1576465003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18440d01b0 0x7f1844147e80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 >> 172.21.15.53:0/1576465003 conn(0x7f184409c3f0 msgr2=0x7f18440b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 shutdown_connections 2023-11-28T11:17:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.733+0000 7f1848dff700 1 -- 172.21.15.53:0/1576465003 wait complete. 2023-11-28T11:17:46.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:17:46.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.953+0000 7fa6f3bff700 1 Processor -- start 2023-11-28T11:17:46.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.953+0000 7fa6f3bff700 1 -- start start 2023-11-28T11:17:46.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.954+0000 7fa6f3bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6ec15c9b0 0x7fa6ec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.954+0000 7fa6f3bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.954+0000 7fa6f3bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0d1950 con 0x7fa6ec161340 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.954+0000 7fa6f3bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b5800 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.954+0000 7fa6f3bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0ce140 con 0x7fa6ec15c9b0 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39632/0 (socket says 172.21.15.53:39632) 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 -- 172.21.15.53:0/3340140779 learned_addr learned my addr 172.21.15.53:0/3340140779 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f199b700 1 --2- 172.21.15.53:0/3340140779 >> v2:172.21.15.157:3300/0 conn(0x7fa6ec15c9b0 0x7fa6ec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 -- 172.21.15.53:0/3340140779 >> v2:172.21.15.157:3300/0 conn(0x7fa6ec15c9b0 msgr2=0x7fa6ec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 --2- 172.21.15.53:0/3340140779 >> v2:172.21.15.157:3300/0 conn(0x7fa6ec15c9b0 0x7fa6ec0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 -- 172.21.15.53:0/3340140779 >> v1:172.21.15.157:6789/0 conn(0x7fa6ec161340 legacy=0x7fa6ec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 -- 172.21.15.53:0/3340140779 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0d24f0 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.955+0000 7fa6f219c700 1 --2- 172.21.15.53:0/3340140779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec067460 tx=0x7fa6e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ed7ac8467a810a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f0999700 1 -- 172.21.15.53:0/3340140779 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400e070 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f0999700 1 -- 172.21.15.53:0/3340140779 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4003a20 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f0999700 1 -- 172.21.15.53:0/3340140779 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4003dd0 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 -- 172.21.15.53:0/3340140779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0cd7c0 msgr2=0x7fa6ec0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/3340140779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec067460 tx=0x7fa6e400b330 comp rx=0 tx=0).stop 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 -- 172.21.15.53:0/3340140779 shutdown_connections 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/3340140779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec0cdc00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/3340140779 >> v2:172.21.15.157:3300/0 conn(0x7fa6ec15c9b0 0x7fa6ec0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 -- 172.21.15.53:0/3340140779 >> 172.21.15.53:0/3340140779 conn(0x7fa6ec072600 msgr2=0x7fa6ec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:46.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 -- 172.21.15.53:0/3340140779 shutdown_connections 2023-11-28T11:17:46.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.956+0000 7fa6f3bff700 1 -- 172.21.15.53:0/3340140779 wait complete. 2023-11-28T11:17:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.958+0000 7fa6f3bff700 1 Processor -- start 2023-11-28T11:17:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.958+0000 7fa6f3bff700 1 -- start start 2023-11-28T11:17:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec15c9b0 0x7fa6ec154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15f0b0 con 0x7fa6ec161340 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec154c60 con 0x7fa6ec0cd7c0 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15e810 con 0x7fa6ec15c9b0 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.959+0000 7fa6f219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39638/0 (socket says 172.21.15.53:39638) 2023-11-28T11:17:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 -- 172.21.15.53:0/4111550222 learned_addr learned my addr 172.21.15.53:0/4111550222 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f119a700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec15c9b0 0x7fa6ec154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f199b700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 -- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec15c9b0 msgr2=0x7fa6ec154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec15c9b0 0x7fa6ec154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 -- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0cd7c0 msgr2=0x7fa6ec1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 -- 172.21.15.53:0/4111550222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009b30 con 0x7fa6ec161340 2023-11-28T11:17:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f219c700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b370 tx=0x7fa6e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004440 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e40092e0 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec0d1300 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.960+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004800 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.961+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0b5520 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.961+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec0d15e0 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.962+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6e4022030 con 0x7fa6ec161340 2023-11-28T11:17:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.963+0000 7fa6e27fc700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 0x7fa6d006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.963+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa6e4086b90 con 0x7fa6ec161340 2023-11-28T11:17:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.964+0000 7fa6f199b700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 0x7fa6d006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.964+0000 7fa6f199b700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 0x7fa6d006a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006b20 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:46.967+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6e4053650 con 0x7fa6ec161340 2023-11-28T11:17:47.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.172+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa6ec0745c0 con 0x7fa6ec161340 2023-11-28T11:17:47.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.172+0000 7fa6e27fc700 1 -- 172.21.15.53:0/4111550222 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa6e4017090 con 0x7fa6ec161340 2023-11-28T11:17:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:47.174 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-28T11:17:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:17:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-11-28T11:17:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 2, 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "rbd" 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:17:47.175 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-28T11:17:47.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 msgr2=0x7fa6d006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 0x7fa6d006a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006b20 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2023-11-28T11:17:47.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 msgr2=0x7fa6ec15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b370 tx=0x7fa6e4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 shutdown_connections 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d0068320 0x7fa6d006a7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec161340 0x7fa6ec15dff0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0cd7c0 0x7fa6ec1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 --2- 172.21.15.53:0/4111550222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec15c9b0 0x7fa6ec154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 >> 172.21.15.53:0/4111550222 conn(0x7fa6ec072600 msgr2=0x7fa6ec0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.174+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 shutdown_connections 2023-11-28T11:17:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.175+0000 7fa6f3bff700 1 -- 172.21.15.53:0/4111550222 wait complete. 2023-11-28T11:17:47.194 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:17:47.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:17:47.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.398+0000 7f12b3bff700 1 Processor -- start 2023-11-28T11:17:47.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.399+0000 7f12b3bff700 1 -- start start 2023-11-28T11:17:47.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12ac15a760 0x7f12ac161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 0x7f12ac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac0cd7c0 0x7f12ac0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac0ce180 con 0x7f12ac15a760 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac15ab30 con 0x7f12ac15dc30 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac15e0a0 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12ac15a760 0x7f12ac161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12ac15a760 0x7f12ac161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39640/0 (socket says 172.21.15.53:39640) 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 -- 172.21.15.53:0/1228118965 learned_addr learned my addr 172.21.15.53:0/1228118965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b219c700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac0cd7c0 0x7f12ac0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b119a700 1 --2- 172.21.15.53:0/1228118965 >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 0x7f12ac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 -- 172.21.15.53:0/1228118965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac0cd7c0 msgr2=0x7f12ac0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac0cd7c0 0x7f12ac0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 -- 172.21.15.53:0/1228118965 >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 msgr2=0x7f12ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:47.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 --2- 172.21.15.53:0/1228118965 >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 0x7f12ac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b199b700 1 -- 172.21.15.53:0/1228118965 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac160d70 con 0x7f12ac15a760 2023-11-28T11:17:47.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.400+0000 7f12b119a700 1 --2- 172.21.15.53:0/1228118965 >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 0x7f12ac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:47.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b199b700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac15a760 0x7f12ac161200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0d2710 tx=0x7f12a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1680ef6e7e2ceb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b0999700 1 -- 172.21.15.53:0/1228118965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011830 con 0x7f12ac15a760 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b0999700 1 -- 172.21.15.53:0/1228118965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8011990 con 0x7f12ac15a760 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b0999700 1 -- 172.21.15.53:0/1228118965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011d20 con 0x7f12ac15a760 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b3bff700 1 -- 172.21.15.53:0/1228118965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac15a760 msgr2=0x7f12ac161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.401+0000 7f12b3bff700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac15a760 0x7f12ac161200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0d2710 tx=0x7f12a8009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 -- 172.21.15.53:0/1228118965 shutdown_connections 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac0cd7c0 0x7f12ac0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 --2- 172.21.15.53:0/1228118965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac15a760 0x7f12ac161200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 --2- 172.21.15.53:0/1228118965 >> v2:172.21.15.157:3300/0 conn(0x7f12ac15dc30 0x7f12ac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 -- 172.21.15.53:0/1228118965 >> 172.21.15.53:0/1228118965 conn(0x7f12ac072620 msgr2=0x7f12ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:47.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 -- 172.21.15.53:0/1228118965 shutdown_connections 2023-11-28T11:17:47.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.402+0000 7f12b3bff700 1 -- 172.21.15.53:0/1228118965 wait complete. 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.403+0000 7f12b3bff700 1 Processor -- start 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.404+0000 7f12b3bff700 1 -- start start 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.404+0000 7f12b3bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.404+0000 7f12b3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac15a760 0x7f12ac0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 0x7f12ac0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b3bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0caa30 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0cac80 con 0x7f12ac15dc30 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0caf60 con 0x7f12ac15a760 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39652/0 (socket says 172.21.15.53:39652) 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 -- 172.21.15.53:0/4243083607 learned_addr learned my addr 172.21.15.53:0/4243083607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b219c700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 0x7f12ac0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b119a700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac15a760 0x7f12ac0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 -- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac15a760 msgr2=0x7f12ac0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac15a760 0x7f12ac0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 -- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 msgr2=0x7f12ac0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 0x7f12ac0ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 -- 172.21.15.53:0/4243083607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009550 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b219c700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 0x7f12ac0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.405+0000 7f12b199b700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009c40 tx=0x7f12a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.406+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8004300 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.406+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a801f070 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.406+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac0b5390 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.406+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80036a0 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.406+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac0b5130 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.407+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac0b5670 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.408+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f12a802a020 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.408+0000 7f12a27fc700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.409+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f12a8086dd0 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.409+0000 7f12b119a700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.410+0000 7f12b119a700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 0x7f129006a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0d05f0 tx=0x7f129c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.412+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12a8053980 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.615+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f12ac04e620 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.663+0000 7f12a27fc700 1 -- 172.21.15.53:0/4243083607 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f12a8056fa0 con 0x7f12ac0cd7c0 2023-11-28T11:17:47.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.667+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 msgr2=0x7f129006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.667+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 0x7f129006a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0d05f0 tx=0x7f129c008040 comp rx=0 tx=0).stop 2023-11-28T11:17:47.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.667+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 msgr2=0x7f12ac0a9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.667+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009c40 tx=0x7f12a8004750 comp rx=0 tx=0).stop 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 shutdown_connections 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12ac15dc30 0x7f12ac0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ac0cd7c0 0x7f12ac0a9a20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 --2- 172.21.15.53:0/4243083607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12ac15a760 0x7f12ac0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 >> 172.21.15.53:0/4243083607 conn(0x7f12ac072620 msgr2=0x7f12ac0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:47.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.668+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 shutdown_connections 2023-11-28T11:17:47.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.669+0000 7f12b3bff700 1 -- 172.21.15.53:0/4243083607 wait complete. 2023-11-28T11:17:47.672 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-11-28T11:17:47.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:17:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.900+0000 7f849cfff700 1 Processor -- start 2023-11-28T11:17:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.900+0000 7f849cfff700 1 -- start start 2023-11-28T11:17:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 0x7f84980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8498161340 0x7f84980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980b5800 con 0x7f8498161340 2023-11-28T11:17:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980d1950 con 0x7f849815c9b0 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849cfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84980b5520 con 0x7f84980d2320 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8498161340 0x7f84980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8498161340 0x7f84980d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39668/0 (socket says 172.21.15.53:39668) 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 -- 172.21.15.53:0/2409979140 learned_addr learned my addr 172.21.15.53:0/2409979140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849659c700 1 --2- 172.21.15.53:0/2409979140 >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 0x7f84980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f8496d9d700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 -- 172.21.15.53:0/2409979140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 msgr2=0x7f84980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 -- 172.21.15.53:0/2409979140 >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 msgr2=0x7f84980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 --2- 172.21.15.53:0/2409979140 >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 0x7f84980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f848ffff700 1 -- 172.21.15.53:0/2409979140 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980d1300 con 0x7f8498161340 2023-11-28T11:17:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f849659c700 1 --2- 172.21.15.53:0/2409979140 >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 0x7f84980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.902+0000 7f8496d9d700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f848ffff700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8498161340 0x7f84980d1de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8498067460 tx=0x7f8480009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39cce53a8eb2410 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f8495d9b700 1 -- 172.21.15.53:0/2409979140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000b870 con 0x7f8498161340 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f8495d9b700 1 -- 172.21.15.53:0/2409979140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000b9d0 con 0x7f8498161340 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f8495d9b700 1 -- 172.21.15.53:0/2409979140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000bd60 con 0x7f8498161340 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 -- 172.21.15.53:0/2409979140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8498161340 msgr2=0x7f84980d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8498161340 0x7f84980d1de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8498067460 tx=0x7f8480009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 -- 172.21.15.53:0/2409979140 shutdown_connections 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 --2- 172.21.15.53:0/2409979140 >> v2:172.21.15.157:3300/0 conn(0x7f849815c9b0 0x7f84980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 --2- 172.21.15.53:0/2409979140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8498161340 0x7f84980d1de0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 -- 172.21.15.53:0/2409979140 >> 172.21.15.53:0/2409979140 conn(0x7f8498072600 msgr2=0x7f84980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.903+0000 7f849cfff700 1 -- 172.21.15.53:0/2409979140 shutdown_connections 2023-11-28T11:17:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.904+0000 7f849cfff700 1 -- 172.21.15.53:0/2409979140 wait complete. 2023-11-28T11:17:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.905+0000 7f849cfff700 1 Processor -- start 2023-11-28T11:17:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.905+0000 7f849cfff700 1 -- start start 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8498161340 0x7f84981593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980c6730 con 0x7f849815c9b0 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84981f2a00 con 0x7f8498161340 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f849cfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84981f2c90 con 0x7f84980d2320 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f848ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f848ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39680/0 (socket says 172.21.15.53:39680) 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.906+0000 7f848ffff700 1 -- 172.21.15.53:0/4010277875 learned_addr learned my addr 172.21.15.53:0/4010277875 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f849659c700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f8496d9d700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8498161340 0x7f84981593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 -- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 msgr2=0x7f84980c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 -- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8498161340 msgr2=0x7f84981593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8498161340 0x7f84981593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 -- 172.21.15.53:0/4010277875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480009550 con 0x7f849815c9b0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848ffff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480004540 con 0x7f849815c9b0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84800036a0 con 0x7f849815c9b0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480003a50 con 0x7f849815c9b0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84980b5130 con 0x7f849815c9b0 2023-11-28T11:17:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.907+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84980d15e0 con 0x7f849815c9b0 2023-11-28T11:17:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.909+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84980b52c0 con 0x7f849815c9b0 2023-11-28T11:17:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.910+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f848000f050 con 0x7f849815c9b0 2023-11-28T11:17:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.910+0000 7f848effd700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.910+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8480087480 con 0x7f849815c9b0 2023-11-28T11:17:47.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.911+0000 7f849659c700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:47.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.911+0000 7f849659c700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 0x7f847806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8488009910 tx=0x7f8488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:47.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:47.916+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8480053dd0 con 0x7f849815c9b0 2023-11-28T11:17:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.117+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f84980745c0 con 0x7f849815c9b0 2023-11-28T11:17:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.673+0000 7f848effd700 1 -- 172.21.15.53:0/4010277875 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f84800573f0 con 0x7f849815c9b0 2023-11-28T11:17:48.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.683+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 msgr2=0x7f847806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:48.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.683+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 0x7f847806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8488009910 tx=0x7f8488008040 comp rx=0 tx=0).stop 2023-11-28T11:17:48.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.683+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 msgr2=0x7f8498155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:48.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.683+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480004750 comp rx=0 tx=0).stop 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.690+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 shutdown_connections 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.690+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.690+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84980d2320 0x7f84980c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.690+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8498161340 0x7f84981593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.690+0000 7f849cfff700 1 --2- 172.21.15.53:0/4010277875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f849815c9b0 0x7f8498155fe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.691+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 >> 172.21.15.53:0/4010277875 conn(0x7f8498072600 msgr2=0x7f84980b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:48.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.692+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 shutdown_connections 2023-11-28T11:17:48.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.693+0000 7f849cfff700 1 -- 172.21.15.53:0/4010277875 wait complete. 2023-11-28T11:17:48.700 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-11-28T11:17:48.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:17:48.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.923+0000 7fe5f89ff700 1 Processor -- start 2023-11-28T11:17:48.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.923+0000 7fe5f89ff700 1 -- start start 2023-11-28T11:17:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 0x7fe5f40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 0x7fe5f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5f40d2320 0x7fe5f40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40b5800 con 0x7fe5f40d2320 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40d1950 con 0x7fe5f4161340 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.924+0000 7fe5f89ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40b5520 con 0x7fe5f415c9b0 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5f40d2320 0x7fe5f40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5f40d2320 0x7fe5f40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39684/0 (socket says 172.21.15.53:39684) 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 -- 172.21.15.53:0/2970013184 learned_addr learned my addr 172.21.15.53:0/2970013184 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f259c700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 0x7fe5f40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5ebfff700 1 --2- 172.21.15.53:0/2970013184 >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 0x7fe5f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 -- 172.21.15.53:0/2970013184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 msgr2=0x7fe5f40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 0x7fe5f40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 -- 172.21.15.53:0/2970013184 >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 msgr2=0x7fe5f40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 --2- 172.21.15.53:0/2970013184 >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 0x7fe5f40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 -- 172.21.15.53:0/2970013184 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40d1300 con 0x7fe5f40d2320 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f259c700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 0x7fe5f40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5ebfff700 1 --2- 172.21.15.53:0/2970013184 >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 0x7fe5f40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.925+0000 7fe5f2d9d700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f40d2320 0x7fe5f40cd7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5f406bc00 tx=0x7fe5ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b4daec944ffb5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f1d9b700 1 -- 172.21.15.53:0/2970013184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e070 con 0x7fe5f40d2320 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f1d9b700 1 -- 172.21.15.53:0/2970013184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00b9c0 con 0x7fe5f40d2320 2023-11-28T11:17:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f1d9b700 1 -- 172.21.15.53:0/2970013184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00bd70 con 0x7fe5f40d2320 2023-11-28T11:17:48.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 -- 172.21.15.53:0/2970013184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f40d2320 msgr2=0x7fe5f40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:48.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f40d2320 0x7fe5f40cd7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5f406bc00 tx=0x7fe5ec00b330 comp rx=0 tx=0).stop 2023-11-28T11:17:48.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 -- 172.21.15.53:0/2970013184 shutdown_connections 2023-11-28T11:17:48.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f415c9b0 0x7fe5f40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/2970013184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f40d2320 0x7fe5f40cd7c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/2970013184 >> v2:172.21.15.157:3300/0 conn(0x7fe5f4161340 0x7fe5f40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 -- 172.21.15.53:0/2970013184 >> 172.21.15.53:0/2970013184 conn(0x7fe5f4072600 msgr2=0x7fe5f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 -- 172.21.15.53:0/2970013184 shutdown_connections 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.926+0000 7fe5f89ff700 1 -- 172.21.15.53:0/2970013184 wait complete. 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.928+0000 7fe5f89ff700 1 Processor -- start 2023-11-28T11:17:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.928+0000 7fe5f89ff700 1 -- start start 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f40d2320 0x7fe5f414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4161340 0x7fe5f4150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4159440 con 0x7fe5f415c9b0 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f415df20 con 0x7fe5f4161340 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5f89ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4153c00 con 0x7fe5f40d2320 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5ebfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5ebfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39694/0 (socket says 172.21.15.53:39694) 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.929+0000 7fe5ebfff700 1 -- 172.21.15.53:0/3190595213 learned_addr learned my addr 172.21.15.53:0/3190595213 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:48.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5f259c700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f40d2320 0x7fe5f414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5f2d9d700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4161340 0x7fe5f4150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 -- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f40d2320 msgr2=0x7fe5f414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f40d2320 0x7fe5f414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 -- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4161340 msgr2=0x7fe5f4150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4161340 0x7fe5f4150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 -- 172.21.15.53:0/3190595213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009b30 con 0x7fe5f415c9b0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5ebfff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e810 tx=0x7fe5dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc013610 con 0x7fe5f415c9b0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc013770 con 0x7fe5f415c9b0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc013b20 con 0x7fe5f415c9b0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40b5130 con 0x7fe5f415c9b0 2023-11-28T11:17:48.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.930+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f40d15e0 con 0x7fe5f415c9b0 2023-11-28T11:17:48.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.931+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f40b52c0 con 0x7fe5f415c9b0 2023-11-28T11:17:48.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.932+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5dc023070 con 0x7fe5f415c9b0 2023-11-28T11:17:48.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.934+0000 7fe5eaffd700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 0x7fe5d406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:48.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.934+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe5dc085990 con 0x7fe5f415c9b0 2023-11-28T11:17:48.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.934+0000 7fe5f259c700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 0x7fe5d406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.935+0000 7fe5f259c700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 0x7fe5d406a7d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c900 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:48.939+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5dc0520d0 con 0x7fe5f415c9b0 2023-11-28T11:17:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.187+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5f4066260 con 0x7fe5f415c9b0 2023-11-28T11:17:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.193+0000 7fe5eaffd700 1 -- 172.21.15.53:0/3190595213 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe5dc0210c0 con 0x7fe5f415c9b0 2023-11-28T11:17:49.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 msgr2=0x7fe5d406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 0x7fe5d406a7d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c900 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 msgr2=0x7fe5f41501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e810 tx=0x7fe5dc00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 shutdown_connections 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe5d4068320 0x7fe5d406a7d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f40d2320 0x7fe5f414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4161340 0x7fe5f4150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 --2- 172.21.15.53:0/3190595213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5f415c9b0 0x7fe5f41501e0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e810 tx=0x7fe5dc00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 >> 172.21.15.53:0/3190595213 conn(0x7fe5f4072600 msgr2=0x7fe5f40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.202+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 shutdown_connections 2023-11-28T11:17:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.203+0000 7fe5f89ff700 1 -- 172.21.15.53:0/3190595213 wait complete. 2023-11-28T11:17:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:17:49.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:49.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.444+0000 7fe65d9ff700 1 Processor -- start 2023-11-28T11:17:49.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.444+0000 7fe65d9ff700 1 -- start start 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.445+0000 7fe65d9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe65815c9b0 0x7fe6580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.445+0000 7fe65d9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 0x7fe6580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.445+0000 7fe65d9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580d1950 con 0x7fe658161340 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.445+0000 7fe65d9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6580b5800 con 0x7fe65815c9b0 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.445+0000 7fe65d9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580ce140 con 0x7fe6580cd7c0 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe65815c9b0 0x7fe6580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe6567fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe658161340 0x7fe6580d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45232/0 (socket says 172.21.15.53:45232) 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe6567fc700 1 -- 172.21.15.53:0/280389541 learned_addr learned my addr 172.21.15.53:0/280389541 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe6577fe700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 0x7fe6580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe655ffb700 1 -- 172.21.15.53:0/280389541 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709297845 0 0) 0x7fe6580d1950 con 0x7fe658161340 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe655ffb700 1 -- 172.21.15.53:0/280389541 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6400035b0 con 0x7fe658161340 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 -- 172.21.15.53:0/280389541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 msgr2=0x7fe6580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 0x7fe6580cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 -- 172.21.15.53:0/280389541 >> v1:172.21.15.53:6789/0 conn(0x7fe658161340 legacy=0x7fe6580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 -- 172.21.15.53:0/280389541 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580d24f0 con 0x7fe65815c9b0 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe6577fe700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 0x7fe6580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:49.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.446+0000 7fe656ffd700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe65815c9b0 0x7fe6580b4bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe648009840 tx=0x7fe648009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabdb4dd6e676373 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe655ffb700 1 -- 172.21.15.53:0/280389541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648011830 con 0x7fe65815c9b0 2023-11-28T11:17:49.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe655ffb700 1 -- 172.21.15.53:0/280389541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe648011990 con 0x7fe65815c9b0 2023-11-28T11:17:49.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe655ffb700 1 -- 172.21.15.53:0/280389541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648011d20 con 0x7fe65815c9b0 2023-11-28T11:17:49.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 -- 172.21.15.53:0/280389541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe65815c9b0 msgr2=0x7fe6580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe65815c9b0 0x7fe6580b4bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe648009840 tx=0x7fe648009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 -- 172.21.15.53:0/280389541 shutdown_connections 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6580cd7c0 0x7fe6580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/280389541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe65815c9b0 0x7fe6580b4bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 -- 172.21.15.53:0/280389541 >> 172.21.15.53:0/280389541 conn(0x7fe658072600 msgr2=0x7fe6580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 -- 172.21.15.53:0/280389541 shutdown_connections 2023-11-28T11:17:49.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.447+0000 7fe65d9ff700 1 -- 172.21.15.53:0/280389541 wait complete. 2023-11-28T11:17:49.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.450+0000 7fe65d9ff700 1 Processor -- start 2023-11-28T11:17:49.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.450+0000 7fe65d9ff700 1 -- start start 2023-11-28T11:17:49.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 0x7fe658156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65815c9b0 0x7fe65815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c9ff0 con 0x7fe658161340 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580cae60 con 0x7fe6580cd7c0 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe65d9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c9a30 con 0x7fe65815c9b0 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39702/0 (socket says 172.21.15.53:39702) 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.451+0000 7fe6577fe700 1 -- 172.21.15.53:0/1432317003 learned_addr learned my addr 172.21.15.53:0/1432317003 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6567fc700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65815c9b0 0x7fe65815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe656ffd700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 0x7fe658156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 -- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65815c9b0 msgr2=0x7fe65815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65815c9b0 0x7fe65815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 -- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 msgr2=0x7fe658156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 0x7fe658156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 -- 172.21.15.53:0/1432317003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480094a0 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe656ffd700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 0x7fe658156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe6577fe700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe650007e70 tx=0x7fe65000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000de40 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65000f040 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650003c60 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.452+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6580d1300 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.453+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6580b5520 con 0x7fe658161340 2023-11-28T11:17:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.454+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6580d15e0 con 0x7fe658161340 2023-11-28T11:17:49.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.455+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe65001c030 con 0x7fe658161340 2023-11-28T11:17:49.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.455+0000 7fe63ffff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 0x7fe64406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.455+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe650085d70 con 0x7fe658161340 2023-11-28T11:17:49.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.456+0000 7fe656ffd700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 0x7fe64406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.457+0000 7fe656ffd700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 0x7fe64406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6581493a0 tx=0x7fe648008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.459+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe6500524e0 con 0x7fe658161340 2023-11-28T11:17:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.663+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6580745c0 con 0x7fe658161340 2023-11-28T11:17:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.664+0000 7fe63ffff700 1 -- 172.21.15.53:0/1432317003 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17920 (secure 0 0 0) 0x7fe65001a090 con 0x7fe658161340 2023-11-28T11:17:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:49.666 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:49.188347+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:49.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 msgr2=0x7fe64406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 0x7fe64406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6581493a0 tx=0x7fe648008030 comp rx=0 tx=0).stop 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 msgr2=0x7fe65815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe650007e70 tx=0x7fe65000d3b0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 shutdown_connections 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe6440682f0 0x7fe64406a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe658161340 0x7fe65815afc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6580cd7c0 0x7fe658156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 --2- 172.21.15.53:0/1432317003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe65815c9b0 0x7fe65815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 >> 172.21.15.53:0/1432317003 conn(0x7fe658072600 msgr2=0x7fe658160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:49.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 shutdown_connections 2023-11-28T11:17:49.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.667+0000 7fe65d9ff700 1 -- 172.21.15.53:0/1432317003 wait complete. 2023-11-28T11:17:49.689 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:49.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.889+0000 7f1c0cfff700 1 Processor -- start 2023-11-28T11:17:49.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.889+0000 7f1c0cfff700 1 -- start start 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.890+0000 7f1c0cfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c08161340 0x7f1c080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.890+0000 7f1c0cfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c080d2320 0x7f1c080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.890+0000 7f1c0cfff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c080d1950 con 0x7f1c0815c9b0 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.890+0000 7f1c0cfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080b5800 con 0x7f1c080d2320 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.890+0000 7f1c0cfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080b5300 con 0x7f1c08161340 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c080d2320 0x7f1c080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c080d2320 0x7f1c080cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39710/0 (socket says 172.21.15.53:39710) 2023-11-28T11:17:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 -- 172.21.15.53:0/1428184450 learned_addr learned my addr 172.21.15.53:0/1428184450 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c05d9b700 1 --2- 172.21.15.53:0/1428184450 >> v2:172.21.15.157:3300/0 conn(0x7f1c08161340 0x7f1c080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 -- 172.21.15.53:0/1428184450 >> v2:172.21.15.157:3300/0 conn(0x7f1c08161340 msgr2=0x7f1c080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 --2- 172.21.15.53:0/1428184450 >> v2:172.21.15.157:3300/0 conn(0x7f1c08161340 0x7f1c080d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 -- 172.21.15.53:0/1428184450 >> v1:172.21.15.157:6789/0 conn(0x7f1c0815c9b0 legacy=0x7f1c080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 -- 172.21.15.53:0/1428184450 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080b5520 con 0x7f1c080d2320 2023-11-28T11:17:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c06d9d700 1 --2- 172.21.15.53:0/1428184450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c080cd7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c080685c0 tx=0x7f1c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb934d61393689f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c0559a700 1 -- 172.21.15.53:0/1428184450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000e070 con 0x7f1c080d2320 2023-11-28T11:17:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.891+0000 7f1c0559a700 1 -- 172.21.15.53:0/1428184450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00003a20 con 0x7f1c080d2320 2023-11-28T11:17:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0559a700 1 -- 172.21.15.53:0/1428184450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00003dd0 con 0x7f1c080d2320 2023-11-28T11:17:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 -- 172.21.15.53:0/1428184450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 msgr2=0x7f1c080cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/1428184450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c080cd7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c080685c0 tx=0x7f1c0000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 -- 172.21.15.53:0/1428184450 shutdown_connections 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/1428184450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c080cd7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/1428184450 >> v2:172.21.15.157:3300/0 conn(0x7f1c08161340 0x7f1c080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 -- 172.21.15.53:0/1428184450 >> 172.21.15.53:0/1428184450 conn(0x7f1c08072600 msgr2=0x7f1c080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 -- 172.21.15.53:0/1428184450 shutdown_connections 2023-11-28T11:17:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.892+0000 7f1c0cfff700 1 -- 172.21.15.53:0/1428184450 wait complete. 2023-11-28T11:17:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.894+0000 7f1c0cfff700 1 Processor -- start 2023-11-28T11:17:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.894+0000 7f1c0cfff700 1 -- start start 2023-11-28T11:17:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c0815c9b0 0x7f1c08157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08161340 0x7f1c081578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08140a80 con 0x7f1c080d2320 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08143480 con 0x7f1c08161340 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0cfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080c69b0 con 0x7f1c0815c9b0 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39716/0 (socket says 172.21.15.53:39716) 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c0659c700 1 -- 172.21.15.53:0/2495604697 learned_addr learned my addr 172.21.15.53:0/2495604697 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c05d9b700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c0815c9b0 0x7f1c08157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.895+0000 7f1c06d9d700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08161340 0x7f1c081578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 -- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c0815c9b0 msgr2=0x7f1c08157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c0815c9b0 0x7f1c08157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 -- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08161340 msgr2=0x7f1c081578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08161340 0x7f1c081578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 -- 172.21.15.53:0/2495604697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00009b30 con 0x7f1c080d2320 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0659c700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b4a0 tx=0x7f1bf800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8011830 con 0x7f1c080d2320 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8011990 con 0x7f1c080d2320 2023-11-28T11:17:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c080d15e0 con 0x7f1c080d2320 2023-11-28T11:17:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8011d40 con 0x7f1c080d2320 2023-11-28T11:17:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.896+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c080d1300 con 0x7f1c080d2320 2023-11-28T11:17:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.897+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0815cd80 con 0x7f1c080d2320 2023-11-28T11:17:49.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.898+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1bf800f050 con 0x7f1c080d2320 2023-11-28T11:17:49.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.899+0000 7f1bf6ffd700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 0x7f1be806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:49.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.899+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1bf8085960 con 0x7f1c080d2320 2023-11-28T11:17:49.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.899+0000 7f1c05d9b700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 0x7f1be806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:49.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.900+0000 7f1c05d9b700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 0x7f1be806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c081402d0 tx=0x7f1bfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:49.903+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1bf8052070 con 0x7f1c080d2320 2023-11-28T11:17:50.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.108+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c080d08a0 con 0x7f1c080d2320 2023-11-28T11:17:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.109+0000 7f1bf6ffd700 1 -- 172.21.15.53:0/2495604697 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17920 (secure 0 0 0) 0x7f1bf8055690 con 0x7f1c080d2320 2023-11-28T11:17:50.110 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:50.110 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:49.188347+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 msgr2=0x7f1be806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 0x7f1be806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c081402d0 tx=0x7f1bfc008040 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 msgr2=0x7f1c08142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b4a0 tx=0x7f1bf800b860 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 shutdown_connections 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1be80682f0 0x7f1be806a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08161340 0x7f1c081578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c080d2320 0x7f1c08142f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 --2- 172.21.15.53:0/2495604697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c0815c9b0 0x7f1c08157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.111+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 >> 172.21.15.53:0/2495604697 conn(0x7f1c08072600 msgr2=0x7f1c0815bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.112+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 shutdown_connections 2023-11-28T11:17:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.112+0000 7f1c0cfff700 1 -- 172.21.15.53:0/2495604697 wait complete. 2023-11-28T11:17:50.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:17:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.337+0000 7efe5bfff700 1 Processor -- start 2023-11-28T11:17:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.337+0000 7efe5bfff700 1 -- start start 2023-11-28T11:17:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe540b20b0 0x7efe540b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5bfff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540b0230 con 0x7efe540d1330 2023-11-28T11:17:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5bfff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540b4570 con 0x7efe540d01b0 2023-11-28T11:17:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5bfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5415cec0 con 0x7efe540b20b0 2023-11-28T11:17:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5959a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe540d1330 0x7efe540b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45248/0 (socket says 172.21.15.53:45248) 2023-11-28T11:17:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.338+0000 7efe5959a700 1 -- 172.21.15.53:0/368405007 learned_addr learned my addr 172.21.15.53:0/368405007 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe5a59c700 1 --2- 172.21.15.53:0/368405007 >> v2:172.21.15.157:3300/0 conn(0x7efe540b20b0 0x7efe540b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059147114 0 0) 0x7efe540b0230 con 0x7efe540d1330 2023-11-28T11:17:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe400035b0 con 0x7efe540d1330 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe5a59c700 1 -- 172.21.15.53:0/368405007 >> v1:172.21.15.157:6789/0 conn(0x7efe540d01b0 legacy=0x7efe540d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe5a59c700 1 -- 172.21.15.53:0/368405007 >> v1:172.21.15.53:6789/0 conn(0x7efe540d1330 legacy=0x7efe540b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647435662 0 0) 0x7efe400035b0 con 0x7efe540d1330 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe5a59c700 1 -- 172.21.15.53:0/368405007 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540b4850 con 0x7efe540b20b0 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.339+0000 7efe5a59c700 1 --2- 172.21.15.53:0/368405007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540b20b0 0x7efe540b0dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009d90 tx=0x7efe4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8129659f889d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00e070 con 0x7efe540b20b0 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c003a20 con 0x7efe540b20b0 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe58d99700 1 -- 172.21.15.53:0/368405007 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44001670 con 0x7efe540b20b0 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 -- 172.21.15.53:0/368405007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540b20b0 msgr2=0x7efe540b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 --2- 172.21.15.53:0/368405007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540b20b0 0x7efe540b0dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009d90 tx=0x7efe4c00b330 comp rx=0 tx=0).stop 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 -- 172.21.15.53:0/368405007 shutdown_connections 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 --2- 172.21.15.53:0/368405007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540b20b0 0x7efe540b0dd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 -- 172.21.15.53:0/368405007 >> 172.21.15.53:0/368405007 conn(0x7efe5409c3f0 msgr2=0x7efe540d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 -- 172.21.15.53:0/368405007 shutdown_connections 2023-11-28T11:17:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.340+0000 7efe5bfff700 1 -- 172.21.15.53:0/368405007 wait complete. 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.342+0000 7efe5bfff700 1 Processor -- start 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.342+0000 7efe5bfff700 1 -- start start 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 0x7efe540b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540d1330 0x7efe5415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5415b4a0 con 0x7efe540d01b0 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b8730 con 0x7efe540d1330 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5bfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b7bd0 con 0x7efe540b20b0 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5959a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5959a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39728/0 (socket says 172.21.15.53:39728) 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.343+0000 7efe5959a700 1 -- 172.21.15.53:0/1553191645 learned_addr learned my addr 172.21.15.53:0/1553191645 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe59d9b700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 0x7efe540b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5a59c700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540d1330 0x7efe5415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 -- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 msgr2=0x7efe540b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 0x7efe540b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 -- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540d1330 msgr2=0x7efe5415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540d1330 0x7efe5415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 -- 172.21.15.53:0/1553191645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c009a70 con 0x7efe540d01b0 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe59d9b700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 0x7efe540b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5959a700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c7b0 tx=0x7efe4400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44004c80 con 0x7efe540d01b0 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe44004de0 con 0x7efe540d01b0 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe540afec0 con 0x7efe540d01b0 2023-11-28T11:17:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44007310 con 0x7efe540d01b0 2023-11-28T11:17:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.344+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe540afbe0 con 0x7efe540d01b0 2023-11-28T11:17:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.345+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe540b2600 con 0x7efe540d01b0 2023-11-28T11:17:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.346+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe4401a020 con 0x7efe540d01b0 2023-11-28T11:17:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.348+0000 7efe4a7fc700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 0x7efe3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.348+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe44085900 con 0x7efe540d01b0 2023-11-28T11:17:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.348+0000 7efe59d9b700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 0x7efe3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.349+0000 7efe59d9b700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 0x7efe3806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe50000f60 tx=0x7efe50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.352+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe44052120 con 0x7efe540d01b0 2023-11-28T11:17:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:50.600+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efe540cab30 con 0x7efe540d01b0 2023-11-28T11:17:51.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.198+0000 7efe4a7fc700 1 -- 172.21.15.53:0/1553191645 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7efe540cab30 con 0x7efe540d01b0 2023-11-28T11:17:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 msgr2=0x7efe3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 0x7efe3806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe50000f60 tx=0x7efe50009040 comp rx=0 tx=0).stop 2023-11-28T11:17:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 msgr2=0x7efe540cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c7b0 tx=0x7efe4400cac0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 shutdown_connections 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efe380682f0 0x7efe3806a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe540b20b0 0x7efe540b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540d1330 0x7efe5415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 --2- 172.21.15.53:0/1553191645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe540d01b0 0x7efe540cb7b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 >> 172.21.15.53:0/1553191645 conn(0x7efe5409c3f0 msgr2=0x7efe5406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 shutdown_connections 2023-11-28T11:17:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.201+0000 7efe5bfff700 1 -- 172.21.15.53:0/1553191645 wait complete. 2023-11-28T11:17:51.221 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:17:51.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:17:51.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.426+0000 7f0abddff700 1 Processor -- start 2023-11-28T11:17:51.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.426+0000 7f0abddff700 1 -- start start 2023-11-28T11:17:51.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0abddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 0x7f0ab80b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0abddff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 0x7f0ab80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0abddff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80b0230 con 0x7f0ab80d01b0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0abddff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab815cec0 con 0x7f0ab80b20b0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0abddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80b4570 con 0x7f0ab80d1330 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0ab77fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0ab80d01b0 0x7f0ab80d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46830/0 (socket says 172.21.15.53:46830) 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0ab77fe700 1 -- 172.21.15.53:0/1401414757 learned_addr learned my addr 172.21.15.53:0/1401414757 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.427+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/1401414757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 0x7f0ab80b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab7fff700 1 --2- 172.21.15.53:0/1401414757 >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 0x7f0ab80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182904760 0 0) 0x7f0ab80b0230 con 0x7f0ab80d01b0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa00035b0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070172826 0 0) 0x7f0aa00035b0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/1401414757 >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 msgr2=0x7f0ab80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/1401414757 >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 0x7f0ab80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/1401414757 >> v1:172.21.15.53:6789/0 conn(0x7f0ab80d01b0 legacy=0x7f0ab80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/1401414757 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80b4850 con 0x7f0ab80d1330 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab7fff700 1 --2- 172.21.15.53:0/1401414757 >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 0x7f0ab80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.428+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/1401414757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 0x7f0ab80b06c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009840 tx=0x7f0aac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2651f808859eb690 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8001670 con 0x7f0ab80d1330 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac004030 con 0x7f0ab80d1330 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0ab67fc700 1 -- 172.21.15.53:0/1401414757 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0043c0 con 0x7f0ab80d1330 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 -- 172.21.15.53:0/1401414757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 msgr2=0x7f0ab80b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 --2- 172.21.15.53:0/1401414757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 0x7f0ab80b06c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009840 tx=0x7f0aac009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 -- 172.21.15.53:0/1401414757 shutdown_connections 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 --2- 172.21.15.53:0/1401414757 >> v2:172.21.15.157:3300/0 conn(0x7f0ab80b20b0 0x7f0ab80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 --2- 172.21.15.53:0/1401414757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80d1330 0x7f0ab80b06c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 -- 172.21.15.53:0/1401414757 >> 172.21.15.53:0/1401414757 conn(0x7f0ab809c3f0 msgr2=0x7f0ab80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 -- 172.21.15.53:0/1401414757 shutdown_connections 2023-11-28T11:17:51.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.429+0000 7f0abddff700 1 -- 172.21.15.53:0/1401414757 wait complete. 2023-11-28T11:17:51.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.431+0000 7f0abddff700 1 Processor -- start 2023-11-28T11:17:51.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.431+0000 7f0abddff700 1 -- start start 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80b20b0 0x7f0ab809f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab80d1330 0x7f0ab80a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab809cd60 con 0x7f0ab80d01b0 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81f30e0 con 0x7f0ab80d1330 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.432+0000 7f0abddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80a08a0 con 0x7f0ab80b20b0 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39804/0 (socket says 172.21.15.53:39804) 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/3035220504 learned_addr learned my addr 172.21.15.53:0/3035220504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab77fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80b20b0 0x7f0ab809f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab7fff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab80d1330 0x7f0ab80a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80b20b0 msgr2=0x7f0ab809f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80b20b0 0x7f0ab809f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab80d1330 msgr2=0x7f0ab80a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab80d1330 0x7f0ab80a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 -- 172.21.15.53:0/3035220504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0009a70 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.433+0000 7f0ab6ffd700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0090b0 tx=0x7f0aac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.434+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003a90 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.434+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac01f070 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.434+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003e50 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.434+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac0094a0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.434+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80afec0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.435+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80afbe0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.436+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0aac02a020 con 0x7f0ab80d01b0 2023-11-28T11:17:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.436+0000 7f0a9ffff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 0x7f0aa406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.437+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0aac0870e0 con 0x7f0ab80d01b0 2023-11-28T11:17:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.437+0000 7f0ab77fe700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 0x7f0aa406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.438+0000 7f0ab77fe700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 0x7f0aa406a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8000f60 tx=0x7f0aa8001230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:51.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.441+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0aac053850 con 0x7f0ab80d01b0 2023-11-28T11:17:51.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:51.690+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0ab80b2600 con 0x7f0ab80d01b0 2023-11-28T11:17:52.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.682+0000 7f0a9ffff700 1 -- 172.21.15.53:0/3035220504 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0aac056e70 con 0x7f0ab80d01b0 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.685+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 msgr2=0x7f0aa406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.685+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 0x7f0aa406a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8000f60 tx=0x7f0aa8001230 comp rx=0 tx=0).stop 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 msgr2=0x7f0ab81f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0090b0 tx=0x7f0aac004750 comp rx=0 tx=0).stop 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 shutdown_connections 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0aa40682f0 0x7f0aa406a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ab80b20b0 0x7f0ab809f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab80d1330 0x7f0ab80a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 --2- 172.21.15.53:0/3035220504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ab80d01b0 0x7f0ab81f2ba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 >> 172.21.15.53:0/3035220504 conn(0x7f0ab809c3f0 msgr2=0x7f0ab80b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 shutdown_connections 2023-11-28T11:17:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.686+0000 7f0abddff700 1 -- 172.21.15.53:0/3035220504 wait complete. 2023-11-28T11:17:52.708 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:17:52.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:17:52.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.914+0000 7f6a181ff700 1 Processor -- start 2023-11-28T11:17:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.915+0000 7f6a181ff700 1 -- start start 2023-11-28T11:17:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a1015c9b0 0x7f6a100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a10161340 0x7f6a100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 0x7f6a100cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100b5800 con 0x7f6a1015c9b0 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100d1950 con 0x7f6a10161340 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.916+0000 7f6a181ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100b5520 con 0x7f6a100d2320 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a1015c9b0 0x7f6a100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a1015c9b0 0x7f6a100b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39806/0 (socket says 172.21.15.53:39806) 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 -- 172.21.15.53:0/3586421613 learned_addr learned my addr 172.21.15.53:0/3586421613 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a1679c700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 0x7f6a100cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a1579a700 1 --2- 172.21.15.53:0/3586421613 >> v2:172.21.15.157:3300/0 conn(0x7f6a10161340 0x7f6a100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 -- 172.21.15.53:0/3586421613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 msgr2=0x7f6a100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 0x7f6a100cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 -- 172.21.15.53:0/3586421613 >> v2:172.21.15.157:3300/0 conn(0x7f6a10161340 msgr2=0x7f6a100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/3586421613 >> v2:172.21.15.157:3300/0 conn(0x7f6a10161340 0x7f6a100d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 -- 172.21.15.53:0/3586421613 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a100d1300 con 0x7f6a1015c9b0 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a1679c700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 0x7f6a100cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.917+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015c9b0 0x7f6a100b4bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a100685c0 tx=0x7f6a0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d4ffe84c8764d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a14f99700 1 -- 172.21.15.53:0/3586421613 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00b870 con 0x7f6a1015c9b0 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a14f99700 1 -- 172.21.15.53:0/3586421613 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00b9d0 con 0x7f6a1015c9b0 2023-11-28T11:17:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a14f99700 1 -- 172.21.15.53:0/3586421613 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00bd60 con 0x7f6a1015c9b0 2023-11-28T11:17:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 -- 172.21.15.53:0/3586421613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015c9b0 msgr2=0x7f6a100b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015c9b0 0x7f6a100b4bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a100685c0 tx=0x7f6a0c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 -- 172.21.15.53:0/3586421613 shutdown_connections 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a100d2320 0x7f6a100cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 --2- 172.21.15.53:0/3586421613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015c9b0 0x7f6a100b4bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 --2- 172.21.15.53:0/3586421613 >> v2:172.21.15.157:3300/0 conn(0x7f6a10161340 0x7f6a100d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 -- 172.21.15.53:0/3586421613 >> 172.21.15.53:0/3586421613 conn(0x7f6a10072600 msgr2=0x7f6a100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 -- 172.21.15.53:0/3586421613 shutdown_connections 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.918+0000 7f6a181ff700 1 -- 172.21.15.53:0/3586421613 wait complete. 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.920+0000 7f6a181ff700 1 Processor -- start 2023-11-28T11:17:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.920+0000 7f6a181ff700 1 -- start start 2023-11-28T11:17:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a1015c9b0 0x7f6a10155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 0x7f6a1015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10157fb0 con 0x7f6a100d2320 2023-11-28T11:17:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10155020 con 0x7f6a10161340 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a181ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101f5870 con 0x7f6a1015c9b0 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a15f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a15f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39812/0 (socket says 172.21.15.53:39812) 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.921+0000 7f6a15f9b700 1 -- 172.21.15.53:0/228572197 learned_addr learned my addr 172.21.15.53:0/228572197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a1679c700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 0x7f6a1015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a1579a700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a1015c9b0 0x7f6a10155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 -- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a1015c9b0 msgr2=0x7f6a10155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a1015c9b0 0x7f6a10155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 -- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 msgr2=0x7f6a1015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 0x7f6a1015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 -- 172.21.15.53:0/228572197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c009550 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a1679c700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 0x7f6a1015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a15f9b700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c000c00 tx=0x7f6a0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004540 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c0036a0 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a100b5130 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c003a50 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.922+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a100d15e0 con 0x7f6a100d2320 2023-11-28T11:17:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.923+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a100b52c0 con 0x7f6a100d2320 2023-11-28T11:17:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.924+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a0c00f050 con 0x7f6a100d2320 2023-11-28T11:17:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.925+0000 7f6a067fc700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 0x7f69f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.925+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a0c087610 con 0x7f6a100d2320 2023-11-28T11:17:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.926+0000 7f6a1579a700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 0x7f69f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.926+0000 7f6a1579a700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 0x7f69f406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d8d0 tx=0x7f6a00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:52.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:52.930+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a0c053d80 con 0x7f6a100d2320 2023-11-28T11:17:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.178+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a100745c0 con 0x7f6a100d2320 2023-11-28T11:17:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.178+0000 7f6a067fc700 1 -- 172.21.15.53:0/228572197 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7f6a0c0573a0 con 0x7f6a100d2320 2023-11-28T11:17:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:53.180 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 msgr2=0x7f69f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 0x7f69f406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d8d0 tx=0x7f6a00008040 comp rx=0 tx=0).stop 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 msgr2=0x7f6a10157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c000c00 tx=0x7f6a0c004750 comp rx=0 tx=0).stop 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 shutdown_connections 2023-11-28T11:17:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69f40682f0 0x7f69f406a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a10161340 0x7f6a1015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a100d2320 0x7f6a10157a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 --2- 172.21.15.53:0/228572197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a1015c9b0 0x7f6a10155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 >> 172.21.15.53:0/228572197 conn(0x7f6a10072600 msgr2=0x7f6a1015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 shutdown_connections 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.181+0000 7f6a181ff700 1 -- 172.21.15.53:0/228572197 wait complete. 2023-11-28T11:17:53.183 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:17:53.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:53.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.400+0000 7f3d73bff700 1 Processor -- start 2023-11-28T11:17:53.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.400+0000 7f3d73bff700 1 -- start start 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.401+0000 7f3d73bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.401+0000 7f3d73bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0b20b0 0x7f3d6c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d73bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c0b0230 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d73bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c15cec0 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d73bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b4570 con 0x7f3d6c0b20b0 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39818/0 (socket says 172.21.15.53:39818) 2023-11-28T11:17:53.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 -- 172.21.15.53:0/2315468573 learned_addr learned my addr 172.21.15.53:0/2315468573 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:53.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 -- 172.21.15.53:0/2315468573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0b20b0 msgr2=0x7f3d6c0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:53.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 --2- 172.21.15.53:0/2315468573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0b20b0 0x7f3d6c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 -- 172.21.15.53:0/2315468573 >> v1:172.21.15.157:6789/0 conn(0x7f3d6c0b29b0 legacy=0x7f3d6c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:53.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 -- 172.21.15.53:0/2315468573 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0b4850 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.402+0000 7f3d7199b700 1 --2- 172.21.15.53:0/2315468573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098f0 tx=0x7f3d68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5191c25b2a981992 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d70999700 1 -- 172.21.15.53:0/2315468573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68004230 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d70999700 1 -- 172.21.15.53:0/2315468573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68004390 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d70999700 1 -- 172.21.15.53:0/2315468573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680036a0 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 -- 172.21.15.53:0/2315468573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0d01b0 msgr2=0x7f3d6c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 --2- 172.21.15.53:0/2315468573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098f0 tx=0x7f3d68009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 -- 172.21.15.53:0/2315468573 shutdown_connections 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 --2- 172.21.15.53:0/2315468573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0d01b0 0x7f3d6c0d0c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 --2- 172.21.15.53:0/2315468573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0b20b0 0x7f3d6c0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 -- 172.21.15.53:0/2315468573 >> 172.21.15.53:0/2315468573 conn(0x7f3d6c09c3f0 msgr2=0x7f3d6c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.403+0000 7f3d73bff700 1 -- 172.21.15.53:0/2315468573 shutdown_connections 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.404+0000 7f3d73bff700 1 -- 172.21.15.53:0/2315468573 wait complete. 2023-11-28T11:17:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.405+0000 7f3d73bff700 1 Processor -- start 2023-11-28T11:17:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.405+0000 7f3d73bff700 1 -- start start 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d6c0b20b0 0x7f3d6c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 0x7f3d6c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0ccfe0 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b8150 con 0x7f3d6c0b20b0 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.406+0000 7f3d73bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b8620 con 0x7f3d6c0d01b0 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39820/0 (socket says 172.21.15.53:39820) 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 -- 172.21.15.53:0/3961895940 learned_addr learned my addr 172.21.15.53:0/3961895940 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7199b700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d6c0b20b0 0x7f3d6c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7219c700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 0x7f3d6c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 -- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 msgr2=0x7f3d6c0b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 0x7f3d6c0b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 -- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d6c0b20b0 msgr2=0x7f3d6c15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d6c0b20b0 0x7f3d6c15f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 -- 172.21.15.53:0/3961895940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009550 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7219c700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 0x7f3d6c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d7119a700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b530 tx=0x7f3d5c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.407+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c011830 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.408+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c011990 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.408+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c011d40 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.408+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0afec0 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.408+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c0afbe0 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.409+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c0b2600 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.409+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d5c021070 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.410+0000 7f3d627fc700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.410+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d5c085ce0 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.411+0000 7f3d7199b700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.412+0000 7f3d7199b700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 0x7f3d5006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009130 tx=0x7f3d68008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.414+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d5c0523f0 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.619+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d6c068690 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.620+0000 7f3d627fc700 1 -- 172.21.15.53:0/3961895940 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f3d5c015020 con 0x7f3d6c0b29b0 2023-11-28T11:17:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:53.622 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.622+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 msgr2=0x7f3d5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.622+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 0x7f3d5006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009130 tx=0x7f3d68008030 comp rx=0 tx=0).stop 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.622+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 msgr2=0x7f3d6c0ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b530 tx=0x7f3d5c00b8f0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 shutdown_connections 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d6c0d01b0 0x7f3d6c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d6c0b20b0 0x7f3d6c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 --2- 172.21.15.53:0/3961895940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6c0b29b0 0x7f3d6c0ccaa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 >> 172.21.15.53:0/3961895940 conn(0x7f3d6c09c3f0 msgr2=0x7f3d6c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:53.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 shutdown_connections 2023-11-28T11:17:53.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.623+0000 7f3d73bff700 1 -- 172.21.15.53:0/3961895940 wait complete. 2023-11-28T11:17:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:17:53.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:17:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.844+0000 7f598fbff700 1 Processor -- start 2023-11-28T11:17:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.844+0000 7f598fbff700 1 -- start start 2023-11-28T11:17:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.845+0000 7f598fbff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f598815c9b0 0x7f59880b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.845+0000 7f598fbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59880d1950 con 0x7f59880cd7c0 2023-11-28T11:17:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.845+0000 7f598fbff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59880d24f0 con 0x7f5988161340 2023-11-28T11:17:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.845+0000 7f598fbff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59880b5800 con 0x7f598815c9b0 2023-11-28T11:17:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598e19c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f59880cd7c0 0x7f59880b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46834/0 (socket says 172.21.15.53:46834) 2023-11-28T11:17:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598d99b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f598815c9b0 0x7f59880b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598e19c700 1 -- 172.21.15.53:0/961153562 learned_addr learned my addr 172.21.15.53:0/961153562 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598d99b700 1 -- 172.21.15.53:0/961153562 >> v1:172.21.15.157:6789/0 conn(0x7f5988161340 legacy=0x7f59880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598d99b700 1 -- 172.21.15.53:0/961153562 >> v1:172.21.15.53:6789/0 conn(0x7f59880cd7c0 legacy=0x7f59880b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598d99b700 1 -- 172.21.15.53:0/961153562 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880b5520 con 0x7f598815c9b0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598c999700 1 -- 172.21.15.53:0/961153562 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855672843 0 0) 0x7f59880d1950 con 0x7f59880cd7c0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.846+0000 7f598d99b700 1 --2- 172.21.15.53:0/961153562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f598815c9b0 0x7f59880b4bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5984009840 tx=0x7f5984009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f56a00ce2f14c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598c999700 1 -- 172.21.15.53:0/961153562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011830 con 0x7f598815c9b0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598c999700 1 -- 172.21.15.53:0/961153562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984011990 con 0x7f598815c9b0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598c999700 1 -- 172.21.15.53:0/961153562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011d20 con 0x7f598815c9b0 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 -- 172.21.15.53:0/961153562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f598815c9b0 msgr2=0x7f59880b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 --2- 172.21.15.53:0/961153562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f598815c9b0 0x7f59880b4bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5984009840 tx=0x7f5984009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 -- 172.21.15.53:0/961153562 shutdown_connections 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 --2- 172.21.15.53:0/961153562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f598815c9b0 0x7f59880b4bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 -- 172.21.15.53:0/961153562 >> 172.21.15.53:0/961153562 conn(0x7f5988072600 msgr2=0x7f59880cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 -- 172.21.15.53:0/961153562 shutdown_connections 2023-11-28T11:17:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.847+0000 7f598fbff700 1 -- 172.21.15.53:0/961153562 wait complete. 2023-11-28T11:17:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.849+0000 7f598fbff700 1 Processor -- start 2023-11-28T11:17:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.849+0000 7f598fbff700 1 -- start start 2023-11-28T11:17:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59880cd7c0 0x7f59880c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598815c9b0 0x7f5988143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59880c75a0 con 0x7f5988161340 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59880c7880 con 0x7f59880cd7c0 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598fbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59880c7b60 con 0x7f598815c9b0 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598e19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598d99b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59880cd7c0 0x7f59880c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598d19a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598815c9b0 0x7f5988143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.850+0000 7f598e19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39842/0 (socket says 172.21.15.53:39842) 2023-11-28T11:17:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 -- 172.21.15.53:0/2969332515 learned_addr learned my addr 172.21.15.53:0/2969332515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:53.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 -- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598815c9b0 msgr2=0x7f5988143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598815c9b0 0x7f5988143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 -- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59880cd7c0 msgr2=0x7f59880c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59880cd7c0 0x7f59880c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 -- 172.21.15.53:0/2969332515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59840094a0 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598e19c700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f598000cba0 tx=0x7f598000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000aaa0 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598000ac20 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800070d0 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.851+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59880d15e0 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.852+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59880d1300 con 0x7f5988161340 2023-11-28T11:17:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.853+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59880b5130 con 0x7f5988161340 2023-11-28T11:17:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.853+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5980010070 con 0x7f5988161340 2023-11-28T11:17:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.854+0000 7f597e7fc700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 0x7f596c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.854+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59800856d0 con 0x7f5988161340 2023-11-28T11:17:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.855+0000 7f598d99b700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 0x7f596c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.856+0000 7f598d99b700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 0x7f596c06a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59840090b0 tx=0x7f598401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:53.858+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5980051e40 con 0x7f5988161340 2023-11-28T11:17:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.056+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59880745c0 con 0x7f596c0682f0 2023-11-28T11:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.067+0000 7f597e7fc700 1 -- 172.21.15.53:0/2969332515 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286682 (secure 0 0 0) 0x7f59880745c0 con 0x7f596c0682f0 2023-11-28T11:17:54.072 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 msgr2=0x7f596c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 0x7f596c06a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59840090b0 tx=0x7f598401c040 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 msgr2=0x7f5988156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f598000cba0 tx=0x7f598000ceb0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 shutdown_connections 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f596c0682f0 0x7f596c06a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5988161340 0x7f5988156830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59880cd7c0 0x7f59880c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 --2- 172.21.15.53:0/2969332515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f598815c9b0 0x7f5988143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.071+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 >> 172.21.15.53:0/2969332515 conn(0x7f5988072600 msgr2=0x7f59880d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.072+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 shutdown_connections 2023-11-28T11:17:54.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.072+0000 7f598fbff700 1 -- 172.21.15.53:0/2969332515 wait complete. 2023-11-28T11:17:54.076 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:17:54.091 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T11:17:52.932121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":98776,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":95808,"kb_avail":754875944,"statfs":{"total":773094113280,"available":772992966656,"internally_reserved":0,"allocated":2973696,"data_stored":1805735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":98107392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.268227"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220015+0000","last_change":"2023-11-28T11:17:49.213974+0000","last_active":"2023-11-28T11:17:49.220015+0000","last_peered":"2023-11-28T11:17:49.220015+0000","last_clean":"2023-11-28T11:17:49.220015+0000","last_became_active":"2023-11-28T11:17:49.213506+0000","last_became_peered":"2023-11-28T11:17:49.213506+0000","last_unstale":"2023-11-28T11:17:49.220015+0000","last_undegraded":"2023-11-28T11:17:49.220015+0000","last_fullsized":"2023-11-28T11:17:49.220015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199622+0000","last_change":"2023-11-28T11:17:49.228867+0000","last_active":"2023-11-28T11:17:50.199622+0000","last_peered":"2023-11-28T11:17:50.199622+0000","last_clean":"2023-11-28T11:17:50.199622+0000","last_became_active":"2023-11-28T11:17:49.228581+0000","last_became_peered":"2023-11-28T11:17:49.228581+0000","last_unstale":"2023-11-28T11:17:50.199622+0000","last_undegraded":"2023-11-28T11:17:50.199622+0000","last_fullsized":"2023-11-28T11:17:50.199622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:10.291664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200594+0000","last_change":"2023-11-28T11:17:49.223726+0000","last_active":"2023-11-28T11:17:50.200594+0000","last_peered":"2023-11-28T11:17:50.200594+0000","last_clean":"2023-11-28T11:17:50.200594+0000","last_became_active":"2023-11-28T11:17:49.222194+0000","last_became_peered":"2023-11-28T11:17:49.222194+0000","last_unstale":"2023-11-28T11:17:50.200594+0000","last_undegraded":"2023-11-28T11:17:50.200594+0000","last_fullsized":"2023-11-28T11:17:50.200594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:23.103321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217212+0000","last_change":"2023-11-28T11:17:49.214435+0000","last_active":"2023-11-28T11:17:49.217212+0000","last_peered":"2023-11-28T11:17:49.217212+0000","last_clean":"2023-11-28T11:17:49.217212+0000","last_became_active":"2023-11-28T11:17:49.214093+0000","last_became_peered":"2023-11-28T11:17:49.214093+0000","last_unstale":"2023-11-28T11:17:49.217212+0000","last_undegraded":"2023-11-28T11:17:49.217212+0000","last_fullsized":"2023-11-28T11:17:49.217212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221261+0000","last_change":"2023-11-28T11:17:49.217415+0000","last_active":"2023-11-28T11:17:49.221261+0000","last_peered":"2023-11-28T11:17:49.221261+0000","last_clean":"2023-11-28T11:17:49.221261+0000","last_became_active":"2023-11-28T11:17:49.217134+0000","last_became_peered":"2023-11-28T11:17:49.217134+0000","last_unstale":"2023-11-28T11:17:49.221261+0000","last_undegraded":"2023-11-28T11:17:49.221261+0000","last_fullsized":"2023-11-28T11:17:49.221261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200106+0000","last_change":"2023-11-28T11:17:49.208771+0000","last_active":"2023-11-28T11:17:50.200106+0000","last_peered":"2023-11-28T11:17:50.200106+0000","last_clean":"2023-11-28T11:17:50.200106+0000","last_became_active":"2023-11-28T11:17:49.208484+0000","last_became_peered":"2023-11-28T11:17:49.208484+0000","last_unstale":"2023-11-28T11:17:50.200106+0000","last_undegraded":"2023-11-28T11:17:50.200106+0000","last_fullsized":"2023-11-28T11:17:50.200106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:32.183119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224625+0000","last_change":"2023-11-28T11:17:49.220560+0000","last_active":"2023-11-28T11:17:49.224625+0000","last_peered":"2023-11-28T11:17:49.224625+0000","last_clean":"2023-11-28T11:17:49.224625+0000","last_became_active":"2023-11-28T11:17:49.220013+0000","last_became_peered":"2023-11-28T11:17:49.220013+0000","last_unstale":"2023-11-28T11:17:49.224625+0000","last_undegraded":"2023-11-28T11:17:49.224625+0000","last_fullsized":"2023-11-28T11:17:49.224625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199285+0000","last_change":"2023-11-28T11:17:49.207080+0000","last_active":"2023-11-28T11:17:50.199285+0000","last_peered":"2023-11-28T11:17:50.199285+0000","last_clean":"2023-11-28T11:17:50.199285+0000","last_became_active":"2023-11-28T11:17:49.206569+0000","last_became_peered":"2023-11-28T11:17:49.206569+0000","last_unstale":"2023-11-28T11:17:50.199285+0000","last_undegraded":"2023-11-28T11:17:50.199285+0000","last_fullsized":"2023-11-28T11:17:50.199285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:19.859853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220940+0000","last_change":"2023-11-28T11:17:49.214151+0000","last_active":"2023-11-28T11:17:49.220940+0000","last_peered":"2023-11-28T11:17:49.220940+0000","last_clean":"2023-11-28T11:17:49.220940+0000","last_became_active":"2023-11-28T11:17:49.213623+0000","last_became_peered":"2023-11-28T11:17:49.213623+0000","last_unstale":"2023-11-28T11:17:49.220940+0000","last_undegraded":"2023-11-28T11:17:49.220940+0000","last_fullsized":"2023-11-28T11:17:49.220940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199866+0000","last_change":"2023-11-28T11:17:49.223078+0000","last_active":"2023-11-28T11:17:50.199866+0000","last_peered":"2023-11-28T11:17:50.199866+0000","last_clean":"2023-11-28T11:17:50.199866+0000","last_became_active":"2023-11-28T11:17:49.221852+0000","last_became_peered":"2023-11-28T11:17:49.221852+0000","last_unstale":"2023-11-28T11:17:50.199866+0000","last_undegraded":"2023-11-28T11:17:50.199866+0000","last_fullsized":"2023-11-28T11:17:50.199866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:47.568223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200736+0000","last_change":"2023-11-28T11:17:49.223486+0000","last_active":"2023-11-28T11:17:50.200736+0000","last_peered":"2023-11-28T11:17:50.200736+0000","last_clean":"2023-11-28T11:17:50.200736+0000","last_became_active":"2023-11-28T11:17:49.221471+0000","last_became_peered":"2023-11-28T11:17:49.221471+0000","last_unstale":"2023-11-28T11:17:50.200736+0000","last_undegraded":"2023-11-28T11:17:50.200736+0000","last_fullsized":"2023-11-28T11:17:50.200736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.219249+0000","last_change":"2023-11-28T11:17:49.208546+0000","last_active":"2023-11-28T11:17:49.219249+0000","last_peered":"2023-11-28T11:17:49.219249+0000","last_clean":"2023-11-28T11:17:49.219249+0000","last_became_active":"2023-11-28T11:17:49.208144+0000","last_became_peered":"2023-11-28T11:17:49.208144+0000","last_unstale":"2023-11-28T11:17:49.219249+0000","last_undegraded":"2023-11-28T11:17:49.219249+0000","last_fullsized":"2023-11-28T11:17:49.219249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.218198+0000","last_change":"2023-11-28T11:17:49.210677+0000","last_active":"2023-11-28T11:17:49.218198+0000","last_peered":"2023-11-28T11:17:49.218198+0000","last_clean":"2023-11-28T11:17:49.218198+0000","last_became_active":"2023-11-28T11:17:49.210425+0000","last_became_peered":"2023-11-28T11:17:49.210425+0000","last_unstale":"2023-11-28T11:17:49.218198+0000","last_undegraded":"2023-11-28T11:17:49.218198+0000","last_fullsized":"2023-11-28T11:17:49.218198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199673+0000","last_change":"2023-11-28T11:17:49.207731+0000","last_active":"2023-11-28T11:17:50.199673+0000","last_peered":"2023-11-28T11:17:50.199673+0000","last_clean":"2023-11-28T11:17:50.199673+0000","last_became_active":"2023-11-28T11:17:49.207411+0000","last_became_peered":"2023-11-28T11:17:49.207411+0000","last_unstale":"2023-11-28T11:17:50.199673+0000","last_undegraded":"2023-11-28T11:17:50.199673+0000","last_fullsized":"2023-11-28T11:17:50.199673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:57.387438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224650+0000","last_change":"2023-11-28T11:17:49.220742+0000","last_active":"2023-11-28T11:17:49.224650+0000","last_peered":"2023-11-28T11:17:49.224650+0000","last_clean":"2023-11-28T11:17:49.224650+0000","last_became_active":"2023-11-28T11:17:49.220406+0000","last_became_peered":"2023-11-28T11:17:49.220406+0000","last_unstale":"2023-11-28T11:17:49.224650+0000","last_undegraded":"2023-11-28T11:17:49.224650+0000","last_fullsized":"2023-11-28T11:17:49.224650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199468+0000","last_change":"2023-11-28T11:17:49.205686+0000","last_active":"2023-11-28T11:17:50.199468+0000","last_peered":"2023-11-28T11:17:50.199468+0000","last_clean":"2023-11-28T11:17:50.199468+0000","last_became_active":"2023-11-28T11:17:49.205310+0000","last_became_peered":"2023-11-28T11:17:49.205310+0000","last_unstale":"2023-11-28T11:17:50.199468+0000","last_undegraded":"2023-11-28T11:17:50.199468+0000","last_fullsized":"2023-11-28T11:17:50.199468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:23.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220921+0000","last_change":"2023-11-28T11:17:49.213934+0000","last_active":"2023-11-28T11:17:49.220921+0000","last_peered":"2023-11-28T11:17:49.220921+0000","last_clean":"2023-11-28T11:17:49.220921+0000","last_became_active":"2023-11-28T11:17:49.213324+0000","last_became_peered":"2023-11-28T11:17:49.213324+0000","last_unstale":"2023-11-28T11:17:49.220921+0000","last_undegraded":"2023-11-28T11:17:49.220921+0000","last_fullsized":"2023-11-28T11:17:49.220921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200680+0000","last_change":"2023-11-28T11:17:49.207509+0000","last_active":"2023-11-28T11:17:50.200680+0000","last_peered":"2023-11-28T11:17:50.200680+0000","last_clean":"2023-11-28T11:17:50.200680+0000","last_became_active":"2023-11-28T11:17:49.206337+0000","last_became_peered":"2023-11-28T11:17:49.206337+0000","last_unstale":"2023-11-28T11:17:50.200680+0000","last_undegraded":"2023-11-28T11:17:50.200680+0000","last_fullsized":"2023-11-28T11:17:50.200680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.219986+0000","last_change":"2023-11-28T11:17:49.213657+0000","last_active":"2023-11-28T11:17:49.219986+0000","last_peered":"2023-11-28T11:17:49.219986+0000","last_clean":"2023-11-28T11:17:49.219986+0000","last_became_active":"2023-11-28T11:17:49.213294+0000","last_became_peered":"2023-11-28T11:17:49.213294+0000","last_unstale":"2023-11-28T11:17:49.219986+0000","last_undegraded":"2023-11-28T11:17:49.219986+0000","last_fullsized":"2023-11-28T11:17:49.219986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:50.362636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200525+0000","last_change":"2023-11-28T11:17:49.211522+0000","last_active":"2023-11-28T11:17:50.200525+0000","last_peered":"2023-11-28T11:17:50.200525+0000","last_clean":"2023-11-28T11:17:50.200525+0000","last_became_active":"2023-11-28T11:17:49.211218+0000","last_became_peered":"2023-11-28T11:17:49.211218+0000","last_unstale":"2023-11-28T11:17:50.200525+0000","last_undegraded":"2023-11-28T11:17:50.200525+0000","last_fullsized":"2023-11-28T11:17:50.200525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:54.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225443+0000","last_change":"2023-11-28T11:17:49.223624+0000","last_active":"2023-11-28T11:17:49.225443+0000","last_peered":"2023-11-28T11:17:49.225443+0000","last_clean":"2023-11-28T11:17:49.225443+0000","last_became_active":"2023-11-28T11:17:49.222927+0000","last_became_peered":"2023-11-28T11:17:49.222927+0000","last_unstale":"2023-11-28T11:17:49.225443+0000","last_undegraded":"2023-11-28T11:17:49.225443+0000","last_fullsized":"2023-11-28T11:17:49.225443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227882+0000","last_change":"2023-11-28T11:17:49.225717+0000","last_active":"2023-11-28T11:17:49.227882+0000","last_peered":"2023-11-28T11:17:49.227882+0000","last_clean":"2023-11-28T11:17:49.227882+0000","last_became_active":"2023-11-28T11:17:49.225457+0000","last_became_peered":"2023-11-28T11:17:49.225457+0000","last_unstale":"2023-11-28T11:17:49.227882+0000","last_undegraded":"2023-11-28T11:17:49.227882+0000","last_fullsized":"2023-11-28T11:17:49.227882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217701+0000","last_change":"2023-11-28T11:17:49.214153+0000","last_active":"2023-11-28T11:17:49.217701+0000","last_peered":"2023-11-28T11:17:49.217701+0000","last_clean":"2023-11-28T11:17:49.217701+0000","last_became_active":"2023-11-28T11:17:49.213231+0000","last_became_peered":"2023-11-28T11:17:49.213231+0000","last_unstale":"2023-11-28T11:17:49.217701+0000","last_undegraded":"2023-11-28T11:17:49.217701+0000","last_fullsized":"2023-11-28T11:17:49.217701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223521+0000","last_change":"2023-11-28T11:17:49.219658+0000","last_active":"2023-11-28T11:17:49.223521+0000","last_peered":"2023-11-28T11:17:49.223521+0000","last_clean":"2023-11-28T11:17:49.223521+0000","last_became_active":"2023-11-28T11:17:49.219357+0000","last_became_peered":"2023-11-28T11:17:49.219357+0000","last_unstale":"2023-11-28T11:17:49.223521+0000","last_undegraded":"2023-11-28T11:17:49.223521+0000","last_fullsized":"2023-11-28T11:17:49.223521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198788+0000","last_change":"2023-11-28T11:17:49.224287+0000","last_active":"2023-11-28T11:17:50.198788+0000","last_peered":"2023-11-28T11:17:50.198788+0000","last_clean":"2023-11-28T11:17:50.198788+0000","last_became_active":"2023-11-28T11:17:49.223202+0000","last_became_peered":"2023-11-28T11:17:49.223202+0000","last_unstale":"2023-11-28T11:17:50.198788+0000","last_undegraded":"2023-11-28T11:17:50.198788+0000","last_fullsized":"2023-11-28T11:17:50.198788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198664+0000","last_change":"2023-11-28T11:17:49.223865+0000","last_active":"2023-11-28T11:17:50.198664+0000","last_peered":"2023-11-28T11:17:50.198664+0000","last_clean":"2023-11-28T11:17:50.198664+0000","last_became_active":"2023-11-28T11:17:49.223609+0000","last_became_peered":"2023-11-28T11:17:49.223609+0000","last_unstale":"2023-11-28T11:17:50.198664+0000","last_undegraded":"2023-11-28T11:17:50.198664+0000","last_fullsized":"2023-11-28T11:17:50.198664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199744+0000","last_change":"2023-11-28T11:17:49.223914+0000","last_active":"2023-11-28T11:17:50.199744+0000","last_peered":"2023-11-28T11:17:50.199744+0000","last_clean":"2023-11-28T11:17:50.199744+0000","last_became_active":"2023-11-28T11:17:49.222584+0000","last_became_peered":"2023-11-28T11:17:49.222584+0000","last_unstale":"2023-11-28T11:17:50.199744+0000","last_undegraded":"2023-11-28T11:17:50.199744+0000","last_fullsized":"2023-11-28T11:17:50.199744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:52.447012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200227+0000","last_change":"2023-11-28T11:17:48.699811+0000","last_active":"2023-11-28T11:17:50.200227+0000","last_peered":"2023-11-28T11:17:50.200227+0000","last_clean":"2023-11-28T11:17:50.200227+0000","last_became_active":"2023-11-28T11:17:48.699476+0000","last_became_peered":"2023-11-28T11:17:48.699476+0000","last_unstale":"2023-11-28T11:17:50.200227+0000","last_undegraded":"2023-11-28T11:17:50.200227+0000","last_fullsized":"2023-11-28T11:17:50.200227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212805+0000","last_change":"2023-11-28T11:17:49.209310+0000","last_active":"2023-11-28T11:17:49.212805+0000","last_peered":"2023-11-28T11:17:49.212805+0000","last_clean":"2023-11-28T11:17:49.212805+0000","last_became_active":"2023-11-28T11:17:49.209040+0000","last_became_peered":"2023-11-28T11:17:49.209040+0000","last_unstale":"2023-11-28T11:17:49.212805+0000","last_undegraded":"2023-11-28T11:17:49.212805+0000","last_fullsized":"2023-11-28T11:17:49.212805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198161+0000","last_change":"2023-11-28T11:17:48.685181+0000","last_active":"2023-11-28T11:17:50.198161+0000","last_peered":"2023-11-28T11:17:50.198161+0000","last_clean":"2023-11-28T11:17:50.198161+0000","last_became_active":"2023-11-28T11:17:48.684843+0000","last_became_peered":"2023-11-28T11:17:48.684843+0000","last_unstale":"2023-11-28T11:17:50.198161+0000","last_undegraded":"2023-11-28T11:17:50.198161+0000","last_fullsized":"2023-11-28T11:17:50.198161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227494+0000","last_change":"2023-11-28T11:17:49.224963+0000","last_active":"2023-11-28T11:17:49.227494+0000","last_peered":"2023-11-28T11:17:49.227494+0000","last_clean":"2023-11-28T11:17:49.227494+0000","last_became_active":"2023-11-28T11:17:49.224679+0000","last_became_peered":"2023-11-28T11:17:49.224679+0000","last_unstale":"2023-11-28T11:17:49.227494+0000","last_undegraded":"2023-11-28T11:17:49.227494+0000","last_fullsized":"2023-11-28T11:17:49.227494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200107+0000","last_change":"2023-11-28T11:17:48.690687+0000","last_active":"2023-11-28T11:17:49.200107+0000","last_peered":"2023-11-28T11:17:49.200107+0000","last_clean":"2023-11-28T11:17:49.200107+0000","last_became_active":"2023-11-28T11:17:48.690509+0000","last_became_peered":"2023-11-28T11:17:48.690509+0000","last_unstale":"2023-11-28T11:17:49.200107+0000","last_undegraded":"2023-11-28T11:17:49.200107+0000","last_fullsized":"2023-11-28T11:17:49.200107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217149+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:49.217149+0000","last_peered":"2023-11-28T11:17:49.217149+0000","last_clean":"2023-11-28T11:17:49.217149+0000","last_became_active":"2023-11-28T11:17:49.215446+0000","last_became_peered":"2023-11-28T11:17:49.215446+0000","last_unstale":"2023-11-28T11:17:49.217149+0000","last_undegraded":"2023-11-28T11:17:49.217149+0000","last_fullsized":"2023-11-28T11:17:49.217149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.206127+0000","last_change":"2023-11-28T11:17:48.689522+0000","last_active":"2023-11-28T11:17:49.206127+0000","last_peered":"2023-11-28T11:17:49.206127+0000","last_clean":"2023-11-28T11:17:49.206127+0000","last_became_active":"2023-11-28T11:17:48.689211+0000","last_became_peered":"2023-11-28T11:17:48.689211+0000","last_unstale":"2023-11-28T11:17:49.206127+0000","last_undegraded":"2023-11-28T11:17:49.206127+0000","last_fullsized":"2023-11-28T11:17:49.206127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225099+0000","last_change":"2023-11-28T11:17:49.222548+0000","last_active":"2023-11-28T11:17:49.225099+0000","last_peered":"2023-11-28T11:17:49.225099+0000","last_clean":"2023-11-28T11:17:49.225099+0000","last_became_active":"2023-11-28T11:17:49.222341+0000","last_became_peered":"2023-11-28T11:17:49.222341+0000","last_unstale":"2023-11-28T11:17:49.225099+0000","last_undegraded":"2023-11-28T11:17:49.225099+0000","last_fullsized":"2023-11-28T11:17:49.225099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199728+0000","last_change":"2023-11-28T11:17:48.696385+0000","last_active":"2023-11-28T11:17:50.199728+0000","last_peered":"2023-11-28T11:17:50.199728+0000","last_clean":"2023-11-28T11:17:50.199728+0000","last_became_active":"2023-11-28T11:17:48.696028+0000","last_became_peered":"2023-11-28T11:17:48.696028+0000","last_unstale":"2023-11-28T11:17:50.199728+0000","last_undegraded":"2023-11-28T11:17:50.199728+0000","last_fullsized":"2023-11-28T11:17:50.199728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:49.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.214215+0000","last_change":"2023-11-28T11:17:49.210958+0000","last_active":"2023-11-28T11:17:49.214215+0000","last_peered":"2023-11-28T11:17:49.214215+0000","last_clean":"2023-11-28T11:17:49.214215+0000","last_became_active":"2023-11-28T11:17:49.210651+0000","last_became_peered":"2023-11-28T11:17:49.210651+0000","last_unstale":"2023-11-28T11:17:49.214215+0000","last_undegraded":"2023-11-28T11:17:49.214215+0000","last_fullsized":"2023-11-28T11:17:49.214215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224370+0000","last_change":"2023-11-28T11:17:48.697605+0000","last_active":"2023-11-28T11:17:49.224370+0000","last_peered":"2023-11-28T11:17:49.224370+0000","last_clean":"2023-11-28T11:17:49.224370+0000","last_became_active":"2023-11-28T11:17:48.696308+0000","last_became_peered":"2023-11-28T11:17:48.696308+0000","last_unstale":"2023-11-28T11:17:49.224370+0000","last_undegraded":"2023-11-28T11:17:49.224370+0000","last_fullsized":"2023-11-28T11:17:49.224370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:46.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198244+0000","last_change":"2023-11-28T11:17:49.213923+0000","last_active":"2023-11-28T11:17:50.198244+0000","last_peered":"2023-11-28T11:17:50.198244+0000","last_clean":"2023-11-28T11:17:50.198244+0000","last_became_active":"2023-11-28T11:17:49.213693+0000","last_became_peered":"2023-11-28T11:17:49.213693+0000","last_unstale":"2023-11-28T11:17:50.198244+0000","last_undegraded":"2023-11-28T11:17:50.198244+0000","last_fullsized":"2023-11-28T11:17:50.198244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200624+0000","last_change":"2023-11-28T11:17:48.693861+0000","last_active":"2023-11-28T11:17:50.200624+0000","last_peered":"2023-11-28T11:17:50.200624+0000","last_clean":"2023-11-28T11:17:50.200624+0000","last_became_active":"2023-11-28T11:17:48.693491+0000","last_became_peered":"2023-11-28T11:17:48.693491+0000","last_unstale":"2023-11-28T11:17:50.200624+0000","last_undegraded":"2023-11-28T11:17:50.200624+0000","last_fullsized":"2023-11-28T11:17:50.200624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200248+0000","last_change":"2023-11-28T11:17:49.213236+0000","last_active":"2023-11-28T11:17:50.200248+0000","last_peered":"2023-11-28T11:17:50.200248+0000","last_clean":"2023-11-28T11:17:50.200248+0000","last_became_active":"2023-11-28T11:17:49.211752+0000","last_became_peered":"2023-11-28T11:17:49.211752+0000","last_unstale":"2023-11-28T11:17:50.200248+0000","last_undegraded":"2023-11-28T11:17:50.200248+0000","last_fullsized":"2023-11-28T11:17:50.200248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:40.807651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226610+0000","last_change":"2023-11-28T11:17:48.702529+0000","last_active":"2023-11-28T11:17:49.226610+0000","last_peered":"2023-11-28T11:17:49.226610+0000","last_clean":"2023-11-28T11:17:49.226610+0000","last_became_active":"2023-11-28T11:17:48.702137+0000","last_became_peered":"2023-11-28T11:17:48.702137+0000","last_unstale":"2023-11-28T11:17:49.226610+0000","last_undegraded":"2023-11-28T11:17:49.226610+0000","last_fullsized":"2023-11-28T11:17:49.226610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:49.562684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225145+0000","last_change":"2023-11-28T11:17:49.223453+0000","last_active":"2023-11-28T11:17:49.225145+0000","last_peered":"2023-11-28T11:17:49.225145+0000","last_clean":"2023-11-28T11:17:49.225145+0000","last_became_active":"2023-11-28T11:17:49.222771+0000","last_became_peered":"2023-11-28T11:17:49.222771+0000","last_unstale":"2023-11-28T11:17:49.225145+0000","last_undegraded":"2023-11-28T11:17:49.225145+0000","last_fullsized":"2023-11-28T11:17:49.225145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202421+0000","last_change":"2023-11-28T11:17:48.689983+0000","last_active":"2023-11-28T11:17:49.202421+0000","last_peered":"2023-11-28T11:17:49.202421+0000","last_clean":"2023-11-28T11:17:49.202421+0000","last_became_active":"2023-11-28T11:17:48.688579+0000","last_became_peered":"2023-11-28T11:17:48.688579+0000","last_unstale":"2023-11-28T11:17:49.202421+0000","last_undegraded":"2023-11-28T11:17:49.202421+0000","last_fullsized":"2023-11-28T11:17:49.202421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199828+0000","last_change":"2023-11-28T11:17:49.219334+0000","last_active":"2023-11-28T11:17:50.199828+0000","last_peered":"2023-11-28T11:17:50.199828+0000","last_clean":"2023-11-28T11:17:50.199828+0000","last_became_active":"2023-11-28T11:17:49.218912+0000","last_became_peered":"2023-11-28T11:17:49.218912+0000","last_unstale":"2023-11-28T11:17:50.199828+0000","last_undegraded":"2023-11-28T11:17:50.199828+0000","last_fullsized":"2023-11-28T11:17:50.199828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:29.752626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200248+0000","last_change":"2023-11-28T11:17:48.702427+0000","last_active":"2023-11-28T11:17:49.200248+0000","last_peered":"2023-11-28T11:17:49.200248+0000","last_clean":"2023-11-28T11:17:49.200248+0000","last_became_active":"2023-11-28T11:17:48.702118+0000","last_became_peered":"2023-11-28T11:17:48.702118+0000","last_unstale":"2023-11-28T11:17:49.200248+0000","last_undegraded":"2023-11-28T11:17:49.200248+0000","last_fullsized":"2023-11-28T11:17:49.200248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:37.239541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221826+0000","last_change":"2023-11-28T11:17:49.209958+0000","last_active":"2023-11-28T11:17:49.221826+0000","last_peered":"2023-11-28T11:17:49.221826+0000","last_clean":"2023-11-28T11:17:49.221826+0000","last_became_active":"2023-11-28T11:17:49.209616+0000","last_became_peered":"2023-11-28T11:17:49.209616+0000","last_unstale":"2023-11-28T11:17:49.221826+0000","last_undegraded":"2023-11-28T11:17:49.221826+0000","last_fullsized":"2023-11-28T11:17:49.221826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.231167+0000","last_change":"2023-11-28T11:17:48.696749+0000","last_active":"2023-11-28T11:17:49.231167+0000","last_peered":"2023-11-28T11:17:49.231167+0000","last_clean":"2023-11-28T11:17:49.231167+0000","last_became_active":"2023-11-28T11:17:48.695400+0000","last_became_peered":"2023-11-28T11:17:48.695400+0000","last_unstale":"2023-11-28T11:17:49.231167+0000","last_undegraded":"2023-11-28T11:17:49.231167+0000","last_fullsized":"2023-11-28T11:17:49.231167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.214383+0000","last_change":"2023-11-28T11:17:49.209165+0000","last_active":"2023-11-28T11:17:49.214383+0000","last_peered":"2023-11-28T11:17:49.214383+0000","last_clean":"2023-11-28T11:17:49.214383+0000","last_became_active":"2023-11-28T11:17:49.208916+0000","last_became_peered":"2023-11-28T11:17:49.208916+0000","last_unstale":"2023-11-28T11:17:49.214383+0000","last_undegraded":"2023-11-28T11:17:49.214383+0000","last_fullsized":"2023-11-28T11:17:49.214383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225371+0000","last_change":"2023-11-28T11:17:48.700934+0000","last_active":"2023-11-28T11:17:49.225371+0000","last_peered":"2023-11-28T11:17:49.225371+0000","last_clean":"2023-11-28T11:17:49.225371+0000","last_became_active":"2023-11-28T11:17:48.700549+0000","last_became_peered":"2023-11-28T11:17:48.700549+0000","last_unstale":"2023-11-28T11:17:49.225371+0000","last_undegraded":"2023-11-28T11:17:49.225371+0000","last_fullsized":"2023-11-28T11:17:49.225371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:40.639240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199331+0000","last_change":"2023-11-28T11:17:49.218909+0000","last_active":"2023-11-28T11:17:50.199331+0000","last_peered":"2023-11-28T11:17:50.199331+0000","last_clean":"2023-11-28T11:17:50.199331+0000","last_became_active":"2023-11-28T11:17:49.218582+0000","last_became_peered":"2023-11-28T11:17:49.218582+0000","last_unstale":"2023-11-28T11:17:50.199331+0000","last_undegraded":"2023-11-28T11:17:50.199331+0000","last_fullsized":"2023-11-28T11:17:50.199331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:25.463820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226887+0000","last_change":"2023-11-28T11:17:48.698986+0000","last_active":"2023-11-28T11:17:49.226887+0000","last_peered":"2023-11-28T11:17:49.226887+0000","last_clean":"2023-11-28T11:17:49.226887+0000","last_became_active":"2023-11-28T11:17:48.698594+0000","last_became_peered":"2023-11-28T11:17:48.698594+0000","last_unstale":"2023-11-28T11:17:49.226887+0000","last_undegraded":"2023-11-28T11:17:49.226887+0000","last_fullsized":"2023-11-28T11:17:49.226887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:19.087105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199992+0000","last_change":"2023-11-28T11:17:49.213092+0000","last_active":"2023-11-28T11:17:50.199992+0000","last_peered":"2023-11-28T11:17:50.199992+0000","last_clean":"2023-11-28T11:17:50.199992+0000","last_became_active":"2023-11-28T11:17:49.211870+0000","last_became_peered":"2023-11-28T11:17:49.211870+0000","last_unstale":"2023-11-28T11:17:50.199992+0000","last_undegraded":"2023-11-28T11:17:50.199992+0000","last_fullsized":"2023-11-28T11:17:50.199992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:41.645456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.228014+0000","last_change":"2023-11-28T11:17:48.695292+0000","last_active":"2023-11-28T11:17:49.228014+0000","last_peered":"2023-11-28T11:17:49.228014+0000","last_clean":"2023-11-28T11:17:49.228014+0000","last_became_active":"2023-11-28T11:17:48.694985+0000","last_became_peered":"2023-11-28T11:17:48.694985+0000","last_unstale":"2023-11-28T11:17:49.228014+0000","last_undegraded":"2023-11-28T11:17:49.228014+0000","last_fullsized":"2023-11-28T11:17:49.228014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:56.077529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227641+0000","last_change":"2023-11-28T11:17:49.226629+0000","last_active":"2023-11-28T11:17:49.227641+0000","last_peered":"2023-11-28T11:17:49.227641+0000","last_clean":"2023-11-28T11:17:49.227641+0000","last_became_active":"2023-11-28T11:17:49.226347+0000","last_became_peered":"2023-11-28T11:17:49.226347+0000","last_unstale":"2023-11-28T11:17:49.227641+0000","last_undegraded":"2023-11-28T11:17:49.227641+0000","last_fullsized":"2023-11-28T11:17:49.227641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199618+0000","last_change":"2023-11-28T11:17:48.692881+0000","last_active":"2023-11-28T11:17:50.199618+0000","last_peered":"2023-11-28T11:17:50.199618+0000","last_clean":"2023-11-28T11:17:50.199618+0000","last_became_active":"2023-11-28T11:17:48.692426+0000","last_became_peered":"2023-11-28T11:17:48.692426+0000","last_unstale":"2023-11-28T11:17:50.199618+0000","last_undegraded":"2023-11-28T11:17:50.199618+0000","last_fullsized":"2023-11-28T11:17:50.199618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:23.247523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226003+0000","last_change":"2023-11-28T11:17:49.221185+0000","last_active":"2023-11-28T11:17:49.226003+0000","last_peered":"2023-11-28T11:17:49.226003+0000","last_clean":"2023-11-28T11:17:49.226003+0000","last_became_active":"2023-11-28T11:17:49.220927+0000","last_became_peered":"2023-11-28T11:17:49.220927+0000","last_unstale":"2023-11-28T11:17:49.226003+0000","last_undegraded":"2023-11-28T11:17:49.226003+0000","last_fullsized":"2023-11-28T11:17:49.226003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224861+0000","last_change":"2023-11-28T11:17:48.698335+0000","last_active":"2023-11-28T11:17:49.224861+0000","last_peered":"2023-11-28T11:17:49.224861+0000","last_clean":"2023-11-28T11:17:49.224861+0000","last_became_active":"2023-11-28T11:17:48.698032+0000","last_became_peered":"2023-11-28T11:17:48.698032+0000","last_unstale":"2023-11-28T11:17:49.224861+0000","last_undegraded":"2023-11-28T11:17:49.224861+0000","last_fullsized":"2023-11-28T11:17:49.224861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200609+0000","last_change":"2023-11-28T11:17:49.207155+0000","last_active":"2023-11-28T11:17:50.200609+0000","last_peered":"2023-11-28T11:17:50.200609+0000","last_clean":"2023-11-28T11:17:50.200609+0000","last_became_active":"2023-11-28T11:17:49.206800+0000","last_became_peered":"2023-11-28T11:17:49.206800+0000","last_unstale":"2023-11-28T11:17:50.200609+0000","last_undegraded":"2023-11-28T11:17:50.200609+0000","last_fullsized":"2023-11-28T11:17:50.200609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.206646+0000","last_change":"2023-11-28T11:17:48.702363+0000","last_active":"2023-11-28T11:17:49.206646+0000","last_peered":"2023-11-28T11:17:49.206646+0000","last_clean":"2023-11-28T11:17:49.206646+0000","last_became_active":"2023-11-28T11:17:48.701778+0000","last_became_peered":"2023-11-28T11:17:48.701778+0000","last_unstale":"2023-11-28T11:17:49.206646+0000","last_undegraded":"2023-11-28T11:17:49.206646+0000","last_fullsized":"2023-11-28T11:17:49.206646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200788+0000","last_change":"2023-11-28T11:17:49.205845+0000","last_active":"2023-11-28T11:17:50.200788+0000","last_peered":"2023-11-28T11:17:50.200788+0000","last_clean":"2023-11-28T11:17:50.200788+0000","last_became_active":"2023-11-28T11:17:49.205302+0000","last_became_peered":"2023-11-28T11:17:49.205302+0000","last_unstale":"2023-11-28T11:17:50.200788+0000","last_undegraded":"2023-11-28T11:17:50.200788+0000","last_fullsized":"2023-11-28T11:17:50.200788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200199+0000","last_change":"2023-11-28T11:17:48.699968+0000","last_active":"2023-11-28T11:17:50.200199+0000","last_peered":"2023-11-28T11:17:50.200199+0000","last_clean":"2023-11-28T11:17:50.200199+0000","last_became_active":"2023-11-28T11:17:48.697897+0000","last_became_peered":"2023-11-28T11:17:48.697897+0000","last_unstale":"2023-11-28T11:17:50.200199+0000","last_undegraded":"2023-11-28T11:17:50.200199+0000","last_fullsized":"2023-11-28T11:17:50.200199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200758+0000","last_change":"2023-11-28T11:17:49.210939+0000","last_active":"2023-11-28T11:17:50.200758+0000","last_peered":"2023-11-28T11:17:50.200758+0000","last_clean":"2023-11-28T11:17:50.200758+0000","last_became_active":"2023-11-28T11:17:49.210585+0000","last_became_peered":"2023-11-28T11:17:49.210585+0000","last_unstale":"2023-11-28T11:17:50.200758+0000","last_undegraded":"2023-11-28T11:17:50.200758+0000","last_fullsized":"2023-11-28T11:17:50.200758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:17.930855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200000+0000","last_change":"2023-11-28T11:17:48.687630+0000","last_active":"2023-11-28T11:17:49.200000+0000","last_peered":"2023-11-28T11:17:49.200000+0000","last_clean":"2023-11-28T11:17:49.200000+0000","last_became_active":"2023-11-28T11:17:48.685494+0000","last_became_peered":"2023-11-28T11:17:48.685494+0000","last_unstale":"2023-11-28T11:17:49.200000+0000","last_undegraded":"2023-11-28T11:17:49.200000+0000","last_fullsized":"2023-11-28T11:17:49.200000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198866+0000","last_change":"2023-11-28T11:17:49.223631+0000","last_active":"2023-11-28T11:17:50.198866+0000","last_peered":"2023-11-28T11:17:50.198866+0000","last_clean":"2023-11-28T11:17:50.198866+0000","last_became_active":"2023-11-28T11:17:49.223321+0000","last_became_peered":"2023-11-28T11:17:49.223321+0000","last_unstale":"2023-11-28T11:17:50.198866+0000","last_undegraded":"2023-11-28T11:17:50.198866+0000","last_fullsized":"2023-11-28T11:17:50.198866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.195384+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:17:49.195384+0000","last_peered":"2023-11-28T11:17:49.195384+0000","last_clean":"2023-11-28T11:17:49.195384+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:17:49.195384+0000","last_undegraded":"2023-11-28T11:17:49.195384+0000","last_fullsized":"2023-11-28T11:17:49.195384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199900+0000","last_change":"2023-11-28T11:17:48.691483+0000","last_active":"2023-11-28T11:17:50.199900+0000","last_peered":"2023-11-28T11:17:50.199900+0000","last_clean":"2023-11-28T11:17:50.199900+0000","last_became_active":"2023-11-28T11:17:48.691194+0000","last_became_peered":"2023-11-28T11:17:48.691194+0000","last_unstale":"2023-11-28T11:17:50.199900+0000","last_undegraded":"2023-11-28T11:17:50.199900+0000","last_fullsized":"2023-11-28T11:17:50.199900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:43.938903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198294+0000","last_change":"2023-11-28T11:17:49.217104+0000","last_active":"2023-11-28T11:17:50.198294+0000","last_peered":"2023-11-28T11:17:50.198294+0000","last_clean":"2023-11-28T11:17:50.198294+0000","last_became_active":"2023-11-28T11:17:49.216883+0000","last_became_peered":"2023-11-28T11:17:49.216883+0000","last_unstale":"2023-11-28T11:17:50.198294+0000","last_undegraded":"2023-11-28T11:17:50.198294+0000","last_fullsized":"2023-11-28T11:17:50.198294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202595+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:17:49.202595+0000","last_peered":"2023-11-28T11:17:49.202595+0000","last_clean":"2023-11-28T11:17:49.202595+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:17:49.202595+0000","last_undegraded":"2023-11-28T11:17:49.202595+0000","last_fullsized":"2023-11-28T11:17:49.202595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200374+0000","last_change":"2023-11-28T11:17:48.699726+0000","last_active":"2023-11-28T11:17:50.200374+0000","last_peered":"2023-11-28T11:17:50.200374+0000","last_clean":"2023-11-28T11:17:50.200374+0000","last_became_active":"2023-11-28T11:17:48.699409+0000","last_became_peered":"2023-11-28T11:17:48.699409+0000","last_unstale":"2023-11-28T11:17:50.200374+0000","last_undegraded":"2023-11-28T11:17:50.200374+0000","last_fullsized":"2023-11-28T11:17:50.200374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198558+0000","last_change":"2023-11-28T11:17:49.223120+0000","last_active":"2023-11-28T11:17:50.198558+0000","last_peered":"2023-11-28T11:17:50.198558+0000","last_clean":"2023-11-28T11:17:50.198558+0000","last_became_active":"2023-11-28T11:17:49.217684+0000","last_became_peered":"2023-11-28T11:17:49.217684+0000","last_unstale":"2023-11-28T11:17:50.198558+0000","last_undegraded":"2023-11-28T11:17:50.198558+0000","last_fullsized":"2023-11-28T11:17:50.198558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.694120+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:17:48.694120+0000","last_peered":"2023-11-28T11:17:48.694120+0000","last_clean":"2023-11-28T11:17:48.694120+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:17:48.694120+0000","last_undegraded":"2023-11-28T11:17:48.694120+0000","last_fullsized":"2023-11-28T11:17:48.694120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198094+0000","last_change":"2023-11-28T11:17:48.695369+0000","last_active":"2023-11-28T11:17:50.198094+0000","last_peered":"2023-11-28T11:17:50.198094+0000","last_clean":"2023-11-28T11:17:50.198094+0000","last_became_active":"2023-11-28T11:17:48.695065+0000","last_became_peered":"2023-11-28T11:17:48.695065+0000","last_unstale":"2023-11-28T11:17:50.198094+0000","last_undegraded":"2023-11-28T11:17:50.198094+0000","last_fullsized":"2023-11-28T11:17:50.198094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198772+0000","last_change":"2023-11-28T11:17:49.217116+0000","last_active":"2023-11-28T11:17:50.198772+0000","last_peered":"2023-11-28T11:17:50.198772+0000","last_clean":"2023-11-28T11:17:50.198772+0000","last_became_active":"2023-11-28T11:17:49.216818+0000","last_became_peered":"2023-11-28T11:17:49.216818+0000","last_unstale":"2023-11-28T11:17:50.198772+0000","last_undegraded":"2023-11-28T11:17:50.198772+0000","last_fullsized":"2023-11-28T11:17:50.198772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202500+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:17:49.202500+0000","last_peered":"2023-11-28T11:17:49.202500+0000","last_clean":"2023-11-28T11:17:49.202500+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:17:49.202500+0000","last_undegraded":"2023-11-28T11:17:49.202500+0000","last_fullsized":"2023-11-28T11:17:49.202500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.686831+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:17:48.686831+0000","last_peered":"2023-11-28T11:17:48.686831+0000","last_clean":"2023-11-28T11:17:48.686831+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:17:48.686831+0000","last_undegraded":"2023-11-28T11:17:48.686831+0000","last_fullsized":"2023-11-28T11:17:48.686831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.197354+0000","last_change":"2023-11-28T11:17:48.692995+0000","last_active":"2023-11-28T11:17:49.197354+0000","last_peered":"2023-11-28T11:17:49.197354+0000","last_clean":"2023-11-28T11:17:49.197354+0000","last_became_active":"2023-11-28T11:17:48.692619+0000","last_became_peered":"2023-11-28T11:17:48.692619+0000","last_unstale":"2023-11-28T11:17:49.197354+0000","last_undegraded":"2023-11-28T11:17:49.197354+0000","last_fullsized":"2023-11-28T11:17:49.197354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212711+0000","last_change":"2023-11-28T11:17:49.211592+0000","last_active":"2023-11-28T11:17:49.212711+0000","last_peered":"2023-11-28T11:17:49.212711+0000","last_clean":"2023-11-28T11:17:49.212711+0000","last_became_active":"2023-11-28T11:17:49.209606+0000","last_became_peered":"2023-11-28T11:17:49.209606+0000","last_unstale":"2023-11-28T11:17:49.212711+0000","last_undegraded":"2023-11-28T11:17:49.212711+0000","last_fullsized":"2023-11-28T11:17:49.212711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:47.444676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.687012+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:17:48.687012+0000","last_peered":"2023-11-28T11:17:48.687012+0000","last_clean":"2023-11-28T11:17:48.687012+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:17:48.687012+0000","last_undegraded":"2023-11-28T11:17:48.687012+0000","last_fullsized":"2023-11-28T11:17:48.687012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202413+0000","last_change":"2023-11-28T11:17:48.696786+0000","last_active":"2023-11-28T11:17:49.202413+0000","last_peered":"2023-11-28T11:17:49.202413+0000","last_clean":"2023-11-28T11:17:49.202413+0000","last_became_active":"2023-11-28T11:17:48.696474+0000","last_became_peered":"2023-11-28T11:17:48.696474+0000","last_unstale":"2023-11-28T11:17:49.202413+0000","last_undegraded":"2023-11-28T11:17:49.202413+0000","last_fullsized":"2023-11-28T11:17:49.202413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227476+0000","last_change":"2023-11-28T11:17:49.219523+0000","last_active":"2023-11-28T11:17:49.227476+0000","last_peered":"2023-11-28T11:17:49.227476+0000","last_clean":"2023-11-28T11:17:49.227476+0000","last_became_active":"2023-11-28T11:17:49.219226+0000","last_became_peered":"2023-11-28T11:17:49.219226+0000","last_unstale":"2023-11-28T11:17:49.227476+0000","last_undegraded":"2023-11-28T11:17:49.227476+0000","last_fullsized":"2023-11-28T11:17:49.227476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.688233+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:17:48.688233+0000","last_peered":"2023-11-28T11:17:48.688233+0000","last_clean":"2023-11-28T11:17:48.688233+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:17:48.688233+0000","last_undegraded":"2023-11-28T11:17:48.688233+0000","last_fullsized":"2023-11-28T11:17:48.688233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198637+0000","last_change":"2023-11-28T11:17:48.696130+0000","last_active":"2023-11-28T11:17:50.198637+0000","last_peered":"2023-11-28T11:17:50.198637+0000","last_clean":"2023-11-28T11:17:50.198637+0000","last_became_active":"2023-11-28T11:17:48.695765+0000","last_became_peered":"2023-11-28T11:17:48.695765+0000","last_unstale":"2023-11-28T11:17:50.198637+0000","last_undegraded":"2023-11-28T11:17:50.198637+0000","last_fullsized":"2023-11-28T11:17:50.198637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223314+0000","last_change":"2023-11-28T11:17:49.219689+0000","last_active":"2023-11-28T11:17:49.223314+0000","last_peered":"2023-11-28T11:17:49.223314+0000","last_clean":"2023-11-28T11:17:49.223314+0000","last_became_active":"2023-11-28T11:17:49.219446+0000","last_became_peered":"2023-11-28T11:17:49.219446+0000","last_unstale":"2023-11-28T11:17:49.223314+0000","last_undegraded":"2023-11-28T11:17:49.223314+0000","last_fullsized":"2023-11-28T11:17:49.223314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224790+0000","last_change":"2023-11-28T11:17:48.694653+0000","last_active":"2023-11-28T11:17:49.224790+0000","last_peered":"2023-11-28T11:17:49.224790+0000","last_clean":"2023-11-28T11:17:49.224790+0000","last_became_active":"2023-11-28T11:17:48.694360+0000","last_became_peered":"2023-11-28T11:17:48.694360+0000","last_unstale":"2023-11-28T11:17:49.224790+0000","last_undegraded":"2023-11-28T11:17:49.224790+0000","last_fullsized":"2023-11-28T11:17:49.224790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.194295+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:17:49.194295+0000","last_peered":"2023-11-28T11:17:49.194295+0000","last_clean":"2023-11-28T11:17:49.194295+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:17:49.194295+0000","last_undegraded":"2023-11-28T11:17:49.194295+0000","last_fullsized":"2023-11-28T11:17:49.194295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223458+0000","last_change":"2023-11-28T11:17:49.219150+0000","last_active":"2023-11-28T11:17:49.223458+0000","last_peered":"2023-11-28T11:17:49.223458+0000","last_clean":"2023-11-28T11:17:49.223458+0000","last_became_active":"2023-11-28T11:17:49.218887+0000","last_became_peered":"2023-11-28T11:17:49.218887+0000","last_unstale":"2023-11-28T11:17:49.223458+0000","last_undegraded":"2023-11-28T11:17:49.223458+0000","last_fullsized":"2023-11-28T11:17:49.223458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.686699+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:17:48.686699+0000","last_peered":"2023-11-28T11:17:48.686699+0000","last_clean":"2023-11-28T11:17:48.686699+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:17:48.686699+0000","last_undegraded":"2023-11-28T11:17:48.686699+0000","last_fullsized":"2023-11-28T11:17:48.686699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.207435+0000","last_change":"2023-11-28T11:17:48.688812+0000","last_active":"2023-11-28T11:17:49.207435+0000","last_peered":"2023-11-28T11:17:49.207435+0000","last_clean":"2023-11-28T11:17:49.207435+0000","last_became_active":"2023-11-28T11:17:48.688551+0000","last_became_peered":"2023-11-28T11:17:48.688551+0000","last_unstale":"2023-11-28T11:17:49.207435+0000","last_undegraded":"2023-11-28T11:17:49.207435+0000","last_fullsized":"2023-11-28T11:17:49.207435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200103+0000","last_change":"2023-11-28T11:17:49.207524+0000","last_active":"2023-11-28T11:17:50.200103+0000","last_peered":"2023-11-28T11:17:50.200103+0000","last_clean":"2023-11-28T11:17:50.200103+0000","last_became_active":"2023-11-28T11:17:49.207175+0000","last_became_peered":"2023-11-28T11:17:49.207175+0000","last_unstale":"2023-11-28T11:17:50.200103+0000","last_undegraded":"2023-11-28T11:17:50.200103+0000","last_fullsized":"2023-11-28T11:17:50.200103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:42.481462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227392+0000","last_change":"2023-11-28T11:17:48.691892+0000","last_active":"2023-11-28T11:17:49.227392+0000","last_peered":"2023-11-28T11:17:49.227392+0000","last_clean":"2023-11-28T11:17:49.227392+0000","last_became_active":"2023-11-28T11:17:48.691619+0000","last_became_peered":"2023-11-28T11:17:48.691619+0000","last_unstale":"2023-11-28T11:17:49.227392+0000","last_undegraded":"2023-11-28T11:17:49.227392+0000","last_fullsized":"2023-11-28T11:17:49.227392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.230934+0000","last_change":"2023-11-28T11:17:49.230012+0000","last_active":"2023-11-28T11:17:49.230934+0000","last_peered":"2023-11-28T11:17:49.230934+0000","last_clean":"2023-11-28T11:17:49.230934+0000","last_became_active":"2023-11-28T11:17:49.229696+0000","last_became_peered":"2023-11-28T11:17:49.229696+0000","last_unstale":"2023-11-28T11:17:49.230934+0000","last_undegraded":"2023-11-28T11:17:49.230934+0000","last_fullsized":"2023-11-28T11:17:49.230934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199712+0000","last_change":"2023-11-28T11:17:48.699672+0000","last_active":"2023-11-28T11:17:50.199712+0000","last_peered":"2023-11-28T11:17:50.199712+0000","last_clean":"2023-11-28T11:17:50.199712+0000","last_became_active":"2023-11-28T11:17:48.699372+0000","last_became_peered":"2023-11-28T11:17:48.699372+0000","last_unstale":"2023-11-28T11:17:50.199712+0000","last_undegraded":"2023-11-28T11:17:50.199712+0000","last_fullsized":"2023-11-28T11:17:50.199712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:53.370355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200423+0000","last_change":"2023-11-28T11:17:49.212871+0000","last_active":"2023-11-28T11:17:50.200423+0000","last_peered":"2023-11-28T11:17:50.200423+0000","last_clean":"2023-11-28T11:17:50.200423+0000","last_became_active":"2023-11-28T11:17:49.212601+0000","last_became_peered":"2023-11-28T11:17:49.212601+0000","last_unstale":"2023-11-28T11:17:50.200423+0000","last_undegraded":"2023-11-28T11:17:50.200423+0000","last_fullsized":"2023-11-28T11:17:50.200423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199934+0000","last_change":"2023-11-28T11:17:48.697131+0000","last_active":"2023-11-28T11:17:50.199934+0000","last_peered":"2023-11-28T11:17:50.199934+0000","last_clean":"2023-11-28T11:17:50.199934+0000","last_became_active":"2023-11-28T11:17:48.696849+0000","last_became_peered":"2023-11-28T11:17:48.696849+0000","last_unstale":"2023-11-28T11:17:50.199934+0000","last_undegraded":"2023-11-28T11:17:50.199934+0000","last_fullsized":"2023-11-28T11:17:50.199934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:09.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200297+0000","last_change":"2023-11-28T11:17:49.216478+0000","last_active":"2023-11-28T11:17:50.200297+0000","last_peered":"2023-11-28T11:17:50.200297+0000","last_clean":"2023-11-28T11:17:50.200297+0000","last_became_active":"2023-11-28T11:17:49.215792+0000","last_became_peered":"2023-11-28T11:17:49.215792+0000","last_unstale":"2023-11-28T11:17:50.200297+0000","last_undegraded":"2023-11-28T11:17:50.200297+0000","last_fullsized":"2023-11-28T11:17:50.200297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220326+0000","last_change":"2023-11-28T11:17:48.692669+0000","last_active":"2023-11-28T11:17:49.220326+0000","last_peered":"2023-11-28T11:17:49.220326+0000","last_clean":"2023-11-28T11:17:49.220326+0000","last_became_active":"2023-11-28T11:17:48.691780+0000","last_became_peered":"2023-11-28T11:17:48.691780+0000","last_unstale":"2023-11-28T11:17:49.220326+0000","last_undegraded":"2023-11-28T11:17:49.220326+0000","last_fullsized":"2023-11-28T11:17:49.220326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.222953+0000","last_change":"2023-11-28T11:17:49.220031+0000","last_active":"2023-11-28T11:17:49.222953+0000","last_peered":"2023-11-28T11:17:49.222953+0000","last_clean":"2023-11-28T11:17:49.222953+0000","last_became_active":"2023-11-28T11:17:49.219784+0000","last_became_peered":"2023-11-28T11:17:49.219784+0000","last_unstale":"2023-11-28T11:17:49.222953+0000","last_undegraded":"2023-11-28T11:17:49.222953+0000","last_fullsized":"2023-11-28T11:17:49.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227804+0000","last_change":"2023-11-28T11:17:48.689149+0000","last_active":"2023-11-28T11:17:49.227804+0000","last_peered":"2023-11-28T11:17:49.227804+0000","last_clean":"2023-11-28T11:17:49.227804+0000","last_became_active":"2023-11-28T11:17:48.688826+0000","last_became_peered":"2023-11-28T11:17:48.688826+0000","last_unstale":"2023-11-28T11:17:49.227804+0000","last_undegraded":"2023-11-28T11:17:49.227804+0000","last_fullsized":"2023-11-28T11:17:49.227804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199319+0000","last_change":"2023-11-28T11:17:49.208765+0000","last_active":"2023-11-28T11:17:50.199319+0000","last_peered":"2023-11-28T11:17:50.199319+0000","last_clean":"2023-11-28T11:17:50.199319+0000","last_became_active":"2023-11-28T11:17:49.208432+0000","last_became_peered":"2023-11-28T11:17:49.208432+0000","last_unstale":"2023-11-28T11:17:50.199319+0000","last_undegraded":"2023-11-28T11:17:50.199319+0000","last_fullsized":"2023-11-28T11:17:50.199319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212723+0000","last_change":"2023-11-28T11:17:49.210834+0000","last_active":"2023-11-28T11:17:49.212723+0000","last_peered":"2023-11-28T11:17:49.212723+0000","last_clean":"2023-11-28T11:17:49.212723+0000","last_became_active":"2023-11-28T11:17:49.208613+0000","last_became_peered":"2023-11-28T11:17:49.208613+0000","last_unstale":"2023-11-28T11:17:49.212723+0000","last_undegraded":"2023-11-28T11:17:49.212723+0000","last_fullsized":"2023-11-28T11:17:49.212723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224638+0000","last_change":"2023-11-28T11:17:49.223484+0000","last_active":"2023-11-28T11:17:49.224638+0000","last_peered":"2023-11-28T11:17:49.224638+0000","last_clean":"2023-11-28T11:17:49.224638+0000","last_became_active":"2023-11-28T11:17:49.222011+0000","last_became_peered":"2023-11-28T11:17:49.222011+0000","last_unstale":"2023-11-28T11:17:49.224638+0000","last_undegraded":"2023-11-28T11:17:49.224638+0000","last_fullsized":"2023-11-28T11:17:49.224638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200166+0000","last_change":"2023-11-28T11:17:49.216669+0000","last_active":"2023-11-28T11:17:50.200166+0000","last_peered":"2023-11-28T11:17:50.200166+0000","last_clean":"2023-11-28T11:17:50.200166+0000","last_became_active":"2023-11-28T11:17:49.216225+0000","last_became_peered":"2023-11-28T11:17:49.216225+0000","last_unstale":"2023-11-28T11:17:50.200166+0000","last_undegraded":"2023-11-28T11:17:50.200166+0000","last_fullsized":"2023-11-28T11:17:50.200166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:22.361832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221248+0000","last_change":"2023-11-28T11:17:49.219093+0000","last_active":"2023-11-28T11:17:49.221248+0000","last_peered":"2023-11-28T11:17:49.221248+0000","last_clean":"2023-11-28T11:17:49.221248+0000","last_became_active":"2023-11-28T11:17:49.218818+0000","last_became_peered":"2023-11-28T11:17:49.218818+0000","last_unstale":"2023-11-28T11:17:49.221248+0000","last_undegraded":"2023-11-28T11:17:49.221248+0000","last_fullsized":"2023-11-28T11:17:49.221248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198448+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.198448+0000","last_peered":"2023-11-28T11:17:50.198448+0000","last_clean":"2023-11-28T11:17:50.198448+0000","last_became_active":"2023-11-28T11:17:49.215424+0000","last_became_peered":"2023-11-28T11:17:49.215424+0000","last_unstale":"2023-11-28T11:17:50.198448+0000","last_undegraded":"2023-11-28T11:17:50.198448+0000","last_fullsized":"2023-11-28T11:17:50.198448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13176,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358664,"statfs":{"total":96636764160,"available":96623271936,"internally_reserved":0,"allocated":835584,"data_stored":668259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11992,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359848,"statfs":{"total":96636764160,"available":96624484352,"internally_reserved":0,"allocated":212992,"data_stored":76127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":81816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13148,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358692,"statfs":{"total":96636764160,"available":96623300608,"internally_reserved":0,"allocated":806912,"data_stored":666042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11992,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359848,"statfs":{"total":96636764160,"available":96624484352,"internally_reserved":0,"allocated":212992,"data_stored":76127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":76217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12140,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359700,"statfs":{"total":96636764160,"available":96624332800,"internally_reserved":0,"allocated":233472,"data_stored":77731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12076,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359764,"statfs":{"total":96636764160,"available":96624398336,"internally_reserved":0,"allocated":233472,"data_stored":83416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:17:54.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:17:54.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.330+0000 7f9690dff700 1 Processor -- start 2023-11-28T11:17:54.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.331+0000 7f9690dff700 1 -- start start 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9690dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 0x7f968c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9690dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9690dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0d1950 con 0x7f968c0d2320 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9690dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c0b5800 con 0x7f968c15c9b0 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9690dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0b5520 con 0x7f968c161340 2023-11-28T11:17:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9683fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f968a59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 0x7f968c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9683fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60136/0 (socket says 172.21.15.53:60136) 2023-11-28T11:17:54.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.332+0000 7f9683fff700 1 -- 172.21.15.53:0/2511550483 learned_addr learned my addr 172.21.15.53:0/2511550483 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:54.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9689d9b700 1 -- 172.21.15.53:0/2511550483 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419196938 0 0) 0x7f968c0d1950 con 0x7f968c0d2320 2023-11-28T11:17:54.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9689d9b700 1 -- 172.21.15.53:0/2511550483 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96780035b0 con 0x7f968c0d2320 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9683fff700 1 -- 172.21.15.53:0/2511550483 >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 msgr2=0x7f968c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9683fff700 1 --2- 172.21.15.53:0/2511550483 >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 0x7f968c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9683fff700 1 -- 172.21.15.53:0/2511550483 >> v1:172.21.15.157:6789/0 conn(0x7f968c0d2320 legacy=0x7f968c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9683fff700 1 -- 172.21.15.53:0/2511550483 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0d1300 con 0x7f968c161340 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f968a59c700 1 --2- 172.21.15.53:0/2511550483 >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 0x7f968c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9683fff700 1 --2- 172.21.15.53:0/2511550483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96740098f0 tx=0x7f9674009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=257f141dde0792c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9689d9b700 1 -- 172.21.15.53:0/2511550483 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674011830 con 0x7f968c161340 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9689d9b700 1 -- 172.21.15.53:0/2511550483 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674011990 con 0x7f968c161340 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.333+0000 7f9689d9b700 1 -- 172.21.15.53:0/2511550483 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674011d20 con 0x7f968c161340 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 -- 172.21.15.53:0/2511550483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 msgr2=0x7f968c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 --2- 172.21.15.53:0/2511550483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96740098f0 tx=0x7f9674009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 -- 172.21.15.53:0/2511550483 shutdown_connections 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 --2- 172.21.15.53:0/2511550483 >> v2:172.21.15.157:3300/0 conn(0x7f968c15c9b0 0x7f968c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 --2- 172.21.15.53:0/2511550483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c161340 0x7f968c0d1de0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 -- 172.21.15.53:0/2511550483 >> 172.21.15.53:0/2511550483 conn(0x7f968c072600 msgr2=0x7f968c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 -- 172.21.15.53:0/2511550483 shutdown_connections 2023-11-28T11:17:54.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.334+0000 7f9690dff700 1 -- 172.21.15.53:0/2511550483 wait complete. 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.336+0000 7f9690dff700 1 Processor -- start 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.336+0000 7f9690dff700 1 -- start start 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c15c9b0 0x7f968c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 0x7f968c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0c9c30 con 0x7f968c0d2320 2023-11-28T11:17:54.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0c7ea0 con 0x7f968c161340 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9690dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c14dfd0 con 0x7f968c15c9b0 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f968a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f968a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39856/0 (socket says 172.21.15.53:39856) 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f968a59c700 1 -- 172.21.15.53:0/4117623021 learned_addr learned my addr 172.21.15.53:0/4117623021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f9683fff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c15c9b0 0x7f968c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.337+0000 7f968ad9d700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 0x7f968c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 -- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c15c9b0 msgr2=0x7f968c0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c15c9b0 0x7f968c0c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 -- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 msgr2=0x7f968c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 0x7f968c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 -- 172.21.15.53:0/4117623021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0094a0 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968ad9d700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 0x7f968c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f968a59c700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ebe0 tx=0x7f967c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c009800 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00c910 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00cc80 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9674009550 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.338+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c0b5130 con 0x7f968c0d2320 2023-11-28T11:17:54.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.339+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c0d15e0 con 0x7f968c0d2320 2023-11-28T11:17:54.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.340+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f967c023070 con 0x7f968c0d2320 2023-11-28T11:17:54.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.342+0000 7f9682ffd700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.342+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f967c086100 con 0x7f968c0d2320 2023-11-28T11:17:54.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.342+0000 7f9683fff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.343+0000 7f9683fff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 0x7f966c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9674009c40 tx=0x7f9674008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.346+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f967c052810 con 0x7f968c0d2320 2023-11-28T11:17:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.542+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f968c0c7fe0 con 0x7f966c0682f0 2023-11-28T11:17:54.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.551+0000 7f9682ffd700 1 -- 172.21.15.53:0/4117623021 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286682 (secure 0 0 0) 0x7f9674011b00 con 0x7f966c0682f0 2023-11-28T11:17:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:54.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 msgr2=0x7f966c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 0x7f966c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9674009c40 tx=0x7f9674008030 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 msgr2=0x7f968c14da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ebe0 tx=0x7f967c00eef0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 shutdown_connections 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f968c161340 0x7f968c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c0d2320 0x7f968c14da90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 --2- 172.21.15.53:0/4117623021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f968c15c9b0 0x7f968c0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 >> 172.21.15.53:0/4117623021 conn(0x7f968c072600 msgr2=0x7f968c15a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 shutdown_connections 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.555+0000 7f9690dff700 1 -- 172.21.15.53:0/4117623021 wait complete. 2023-11-28T11:17:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:17:54.574 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T11:17:52.932121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":98776,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":95808,"kb_avail":754875944,"statfs":{"total":773094113280,"available":772992966656,"internally_reserved":0,"allocated":2973696,"data_stored":1805735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":98107392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.268227"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220015+0000","last_change":"2023-11-28T11:17:49.213974+0000","last_active":"2023-11-28T11:17:49.220015+0000","last_peered":"2023-11-28T11:17:49.220015+0000","last_clean":"2023-11-28T11:17:49.220015+0000","last_became_active":"2023-11-28T11:17:49.213506+0000","last_became_peered":"2023-11-28T11:17:49.213506+0000","last_unstale":"2023-11-28T11:17:49.220015+0000","last_undegraded":"2023-11-28T11:17:49.220015+0000","last_fullsized":"2023-11-28T11:17:49.220015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199622+0000","last_change":"2023-11-28T11:17:49.228867+0000","last_active":"2023-11-28T11:17:50.199622+0000","last_peered":"2023-11-28T11:17:50.199622+0000","last_clean":"2023-11-28T11:17:50.199622+0000","last_became_active":"2023-11-28T11:17:49.228581+0000","last_became_peered":"2023-11-28T11:17:49.228581+0000","last_unstale":"2023-11-28T11:17:50.199622+0000","last_undegraded":"2023-11-28T11:17:50.199622+0000","last_fullsized":"2023-11-28T11:17:50.199622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:10.291664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200594+0000","last_change":"2023-11-28T11:17:49.223726+0000","last_active":"2023-11-28T11:17:50.200594+0000","last_peered":"2023-11-28T11:17:50.200594+0000","last_clean":"2023-11-28T11:17:50.200594+0000","last_became_active":"2023-11-28T11:17:49.222194+0000","last_became_peered":"2023-11-28T11:17:49.222194+0000","last_unstale":"2023-11-28T11:17:50.200594+0000","last_undegraded":"2023-11-28T11:17:50.200594+0000","last_fullsized":"2023-11-28T11:17:50.200594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:23.103321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217212+0000","last_change":"2023-11-28T11:17:49.214435+0000","last_active":"2023-11-28T11:17:49.217212+0000","last_peered":"2023-11-28T11:17:49.217212+0000","last_clean":"2023-11-28T11:17:49.217212+0000","last_became_active":"2023-11-28T11:17:49.214093+0000","last_became_peered":"2023-11-28T11:17:49.214093+0000","last_unstale":"2023-11-28T11:17:49.217212+0000","last_undegraded":"2023-11-28T11:17:49.217212+0000","last_fullsized":"2023-11-28T11:17:49.217212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221261+0000","last_change":"2023-11-28T11:17:49.217415+0000","last_active":"2023-11-28T11:17:49.221261+0000","last_peered":"2023-11-28T11:17:49.221261+0000","last_clean":"2023-11-28T11:17:49.221261+0000","last_became_active":"2023-11-28T11:17:49.217134+0000","last_became_peered":"2023-11-28T11:17:49.217134+0000","last_unstale":"2023-11-28T11:17:49.221261+0000","last_undegraded":"2023-11-28T11:17:49.221261+0000","last_fullsized":"2023-11-28T11:17:49.221261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200106+0000","last_change":"2023-11-28T11:17:49.208771+0000","last_active":"2023-11-28T11:17:50.200106+0000","last_peered":"2023-11-28T11:17:50.200106+0000","last_clean":"2023-11-28T11:17:50.200106+0000","last_became_active":"2023-11-28T11:17:49.208484+0000","last_became_peered":"2023-11-28T11:17:49.208484+0000","last_unstale":"2023-11-28T11:17:50.200106+0000","last_undegraded":"2023-11-28T11:17:50.200106+0000","last_fullsized":"2023-11-28T11:17:50.200106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:32.183119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224625+0000","last_change":"2023-11-28T11:17:49.220560+0000","last_active":"2023-11-28T11:17:49.224625+0000","last_peered":"2023-11-28T11:17:49.224625+0000","last_clean":"2023-11-28T11:17:49.224625+0000","last_became_active":"2023-11-28T11:17:49.220013+0000","last_became_peered":"2023-11-28T11:17:49.220013+0000","last_unstale":"2023-11-28T11:17:49.224625+0000","last_undegraded":"2023-11-28T11:17:49.224625+0000","last_fullsized":"2023-11-28T11:17:49.224625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199285+0000","last_change":"2023-11-28T11:17:49.207080+0000","last_active":"2023-11-28T11:17:50.199285+0000","last_peered":"2023-11-28T11:17:50.199285+0000","last_clean":"2023-11-28T11:17:50.199285+0000","last_became_active":"2023-11-28T11:17:49.206569+0000","last_became_peered":"2023-11-28T11:17:49.206569+0000","last_unstale":"2023-11-28T11:17:50.199285+0000","last_undegraded":"2023-11-28T11:17:50.199285+0000","last_fullsized":"2023-11-28T11:17:50.199285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:19.859853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220940+0000","last_change":"2023-11-28T11:17:49.214151+0000","last_active":"2023-11-28T11:17:49.220940+0000","last_peered":"2023-11-28T11:17:49.220940+0000","last_clean":"2023-11-28T11:17:49.220940+0000","last_became_active":"2023-11-28T11:17:49.213623+0000","last_became_peered":"2023-11-28T11:17:49.213623+0000","last_unstale":"2023-11-28T11:17:49.220940+0000","last_undegraded":"2023-11-28T11:17:49.220940+0000","last_fullsized":"2023-11-28T11:17:49.220940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199866+0000","last_change":"2023-11-28T11:17:49.223078+0000","last_active":"2023-11-28T11:17:50.199866+0000","last_peered":"2023-11-28T11:17:50.199866+0000","last_clean":"2023-11-28T11:17:50.199866+0000","last_became_active":"2023-11-28T11:17:49.221852+0000","last_became_peered":"2023-11-28T11:17:49.221852+0000","last_unstale":"2023-11-28T11:17:50.199866+0000","last_undegraded":"2023-11-28T11:17:50.199866+0000","last_fullsized":"2023-11-28T11:17:50.199866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:47.568223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200736+0000","last_change":"2023-11-28T11:17:49.223486+0000","last_active":"2023-11-28T11:17:50.200736+0000","last_peered":"2023-11-28T11:17:50.200736+0000","last_clean":"2023-11-28T11:17:50.200736+0000","last_became_active":"2023-11-28T11:17:49.221471+0000","last_became_peered":"2023-11-28T11:17:49.221471+0000","last_unstale":"2023-11-28T11:17:50.200736+0000","last_undegraded":"2023-11-28T11:17:50.200736+0000","last_fullsized":"2023-11-28T11:17:50.200736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.219249+0000","last_change":"2023-11-28T11:17:49.208546+0000","last_active":"2023-11-28T11:17:49.219249+0000","last_peered":"2023-11-28T11:17:49.219249+0000","last_clean":"2023-11-28T11:17:49.219249+0000","last_became_active":"2023-11-28T11:17:49.208144+0000","last_became_peered":"2023-11-28T11:17:49.208144+0000","last_unstale":"2023-11-28T11:17:49.219249+0000","last_undegraded":"2023-11-28T11:17:49.219249+0000","last_fullsized":"2023-11-28T11:17:49.219249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.218198+0000","last_change":"2023-11-28T11:17:49.210677+0000","last_active":"2023-11-28T11:17:49.218198+0000","last_peered":"2023-11-28T11:17:49.218198+0000","last_clean":"2023-11-28T11:17:49.218198+0000","last_became_active":"2023-11-28T11:17:49.210425+0000","last_became_peered":"2023-11-28T11:17:49.210425+0000","last_unstale":"2023-11-28T11:17:49.218198+0000","last_undegraded":"2023-11-28T11:17:49.218198+0000","last_fullsized":"2023-11-28T11:17:49.218198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199673+0000","last_change":"2023-11-28T11:17:49.207731+0000","last_active":"2023-11-28T11:17:50.199673+0000","last_peered":"2023-11-28T11:17:50.199673+0000","last_clean":"2023-11-28T11:17:50.199673+0000","last_became_active":"2023-11-28T11:17:49.207411+0000","last_became_peered":"2023-11-28T11:17:49.207411+0000","last_unstale":"2023-11-28T11:17:50.199673+0000","last_undegraded":"2023-11-28T11:17:50.199673+0000","last_fullsized":"2023-11-28T11:17:50.199673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:57.387438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224650+0000","last_change":"2023-11-28T11:17:49.220742+0000","last_active":"2023-11-28T11:17:49.224650+0000","last_peered":"2023-11-28T11:17:49.224650+0000","last_clean":"2023-11-28T11:17:49.224650+0000","last_became_active":"2023-11-28T11:17:49.220406+0000","last_became_peered":"2023-11-28T11:17:49.220406+0000","last_unstale":"2023-11-28T11:17:49.224650+0000","last_undegraded":"2023-11-28T11:17:49.224650+0000","last_fullsized":"2023-11-28T11:17:49.224650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199468+0000","last_change":"2023-11-28T11:17:49.205686+0000","last_active":"2023-11-28T11:17:50.199468+0000","last_peered":"2023-11-28T11:17:50.199468+0000","last_clean":"2023-11-28T11:17:50.199468+0000","last_became_active":"2023-11-28T11:17:49.205310+0000","last_became_peered":"2023-11-28T11:17:49.205310+0000","last_unstale":"2023-11-28T11:17:50.199468+0000","last_undegraded":"2023-11-28T11:17:50.199468+0000","last_fullsized":"2023-11-28T11:17:50.199468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:23.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220921+0000","last_change":"2023-11-28T11:17:49.213934+0000","last_active":"2023-11-28T11:17:49.220921+0000","last_peered":"2023-11-28T11:17:49.220921+0000","last_clean":"2023-11-28T11:17:49.220921+0000","last_became_active":"2023-11-28T11:17:49.213324+0000","last_became_peered":"2023-11-28T11:17:49.213324+0000","last_unstale":"2023-11-28T11:17:49.220921+0000","last_undegraded":"2023-11-28T11:17:49.220921+0000","last_fullsized":"2023-11-28T11:17:49.220921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200680+0000","last_change":"2023-11-28T11:17:49.207509+0000","last_active":"2023-11-28T11:17:50.200680+0000","last_peered":"2023-11-28T11:17:50.200680+0000","last_clean":"2023-11-28T11:17:50.200680+0000","last_became_active":"2023-11-28T11:17:49.206337+0000","last_became_peered":"2023-11-28T11:17:49.206337+0000","last_unstale":"2023-11-28T11:17:50.200680+0000","last_undegraded":"2023-11-28T11:17:50.200680+0000","last_fullsized":"2023-11-28T11:17:50.200680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.219986+0000","last_change":"2023-11-28T11:17:49.213657+0000","last_active":"2023-11-28T11:17:49.219986+0000","last_peered":"2023-11-28T11:17:49.219986+0000","last_clean":"2023-11-28T11:17:49.219986+0000","last_became_active":"2023-11-28T11:17:49.213294+0000","last_became_peered":"2023-11-28T11:17:49.213294+0000","last_unstale":"2023-11-28T11:17:49.219986+0000","last_undegraded":"2023-11-28T11:17:49.219986+0000","last_fullsized":"2023-11-28T11:17:49.219986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:50.362636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200525+0000","last_change":"2023-11-28T11:17:49.211522+0000","last_active":"2023-11-28T11:17:50.200525+0000","last_peered":"2023-11-28T11:17:50.200525+0000","last_clean":"2023-11-28T11:17:50.200525+0000","last_became_active":"2023-11-28T11:17:49.211218+0000","last_became_peered":"2023-11-28T11:17:49.211218+0000","last_unstale":"2023-11-28T11:17:50.200525+0000","last_undegraded":"2023-11-28T11:17:50.200525+0000","last_fullsized":"2023-11-28T11:17:50.200525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:54.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225443+0000","last_change":"2023-11-28T11:17:49.223624+0000","last_active":"2023-11-28T11:17:49.225443+0000","last_peered":"2023-11-28T11:17:49.225443+0000","last_clean":"2023-11-28T11:17:49.225443+0000","last_became_active":"2023-11-28T11:17:49.222927+0000","last_became_peered":"2023-11-28T11:17:49.222927+0000","last_unstale":"2023-11-28T11:17:49.225443+0000","last_undegraded":"2023-11-28T11:17:49.225443+0000","last_fullsized":"2023-11-28T11:17:49.225443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227882+0000","last_change":"2023-11-28T11:17:49.225717+0000","last_active":"2023-11-28T11:17:49.227882+0000","last_peered":"2023-11-28T11:17:49.227882+0000","last_clean":"2023-11-28T11:17:49.227882+0000","last_became_active":"2023-11-28T11:17:49.225457+0000","last_became_peered":"2023-11-28T11:17:49.225457+0000","last_unstale":"2023-11-28T11:17:49.227882+0000","last_undegraded":"2023-11-28T11:17:49.227882+0000","last_fullsized":"2023-11-28T11:17:49.227882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217701+0000","last_change":"2023-11-28T11:17:49.214153+0000","last_active":"2023-11-28T11:17:49.217701+0000","last_peered":"2023-11-28T11:17:49.217701+0000","last_clean":"2023-11-28T11:17:49.217701+0000","last_became_active":"2023-11-28T11:17:49.213231+0000","last_became_peered":"2023-11-28T11:17:49.213231+0000","last_unstale":"2023-11-28T11:17:49.217701+0000","last_undegraded":"2023-11-28T11:17:49.217701+0000","last_fullsized":"2023-11-28T11:17:49.217701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223521+0000","last_change":"2023-11-28T11:17:49.219658+0000","last_active":"2023-11-28T11:17:49.223521+0000","last_peered":"2023-11-28T11:17:49.223521+0000","last_clean":"2023-11-28T11:17:49.223521+0000","last_became_active":"2023-11-28T11:17:49.219357+0000","last_became_peered":"2023-11-28T11:17:49.219357+0000","last_unstale":"2023-11-28T11:17:49.223521+0000","last_undegraded":"2023-11-28T11:17:49.223521+0000","last_fullsized":"2023-11-28T11:17:49.223521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198788+0000","last_change":"2023-11-28T11:17:49.224287+0000","last_active":"2023-11-28T11:17:50.198788+0000","last_peered":"2023-11-28T11:17:50.198788+0000","last_clean":"2023-11-28T11:17:50.198788+0000","last_became_active":"2023-11-28T11:17:49.223202+0000","last_became_peered":"2023-11-28T11:17:49.223202+0000","last_unstale":"2023-11-28T11:17:50.198788+0000","last_undegraded":"2023-11-28T11:17:50.198788+0000","last_fullsized":"2023-11-28T11:17:50.198788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198664+0000","last_change":"2023-11-28T11:17:49.223865+0000","last_active":"2023-11-28T11:17:50.198664+0000","last_peered":"2023-11-28T11:17:50.198664+0000","last_clean":"2023-11-28T11:17:50.198664+0000","last_became_active":"2023-11-28T11:17:49.223609+0000","last_became_peered":"2023-11-28T11:17:49.223609+0000","last_unstale":"2023-11-28T11:17:50.198664+0000","last_undegraded":"2023-11-28T11:17:50.198664+0000","last_fullsized":"2023-11-28T11:17:50.198664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199744+0000","last_change":"2023-11-28T11:17:49.223914+0000","last_active":"2023-11-28T11:17:50.199744+0000","last_peered":"2023-11-28T11:17:50.199744+0000","last_clean":"2023-11-28T11:17:50.199744+0000","last_became_active":"2023-11-28T11:17:49.222584+0000","last_became_peered":"2023-11-28T11:17:49.222584+0000","last_unstale":"2023-11-28T11:17:50.199744+0000","last_undegraded":"2023-11-28T11:17:50.199744+0000","last_fullsized":"2023-11-28T11:17:50.199744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:52.447012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200227+0000","last_change":"2023-11-28T11:17:48.699811+0000","last_active":"2023-11-28T11:17:50.200227+0000","last_peered":"2023-11-28T11:17:50.200227+0000","last_clean":"2023-11-28T11:17:50.200227+0000","last_became_active":"2023-11-28T11:17:48.699476+0000","last_became_peered":"2023-11-28T11:17:48.699476+0000","last_unstale":"2023-11-28T11:17:50.200227+0000","last_undegraded":"2023-11-28T11:17:50.200227+0000","last_fullsized":"2023-11-28T11:17:50.200227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212805+0000","last_change":"2023-11-28T11:17:49.209310+0000","last_active":"2023-11-28T11:17:49.212805+0000","last_peered":"2023-11-28T11:17:49.212805+0000","last_clean":"2023-11-28T11:17:49.212805+0000","last_became_active":"2023-11-28T11:17:49.209040+0000","last_became_peered":"2023-11-28T11:17:49.209040+0000","last_unstale":"2023-11-28T11:17:49.212805+0000","last_undegraded":"2023-11-28T11:17:49.212805+0000","last_fullsized":"2023-11-28T11:17:49.212805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198161+0000","last_change":"2023-11-28T11:17:48.685181+0000","last_active":"2023-11-28T11:17:50.198161+0000","last_peered":"2023-11-28T11:17:50.198161+0000","last_clean":"2023-11-28T11:17:50.198161+0000","last_became_active":"2023-11-28T11:17:48.684843+0000","last_became_peered":"2023-11-28T11:17:48.684843+0000","last_unstale":"2023-11-28T11:17:50.198161+0000","last_undegraded":"2023-11-28T11:17:50.198161+0000","last_fullsized":"2023-11-28T11:17:50.198161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227494+0000","last_change":"2023-11-28T11:17:49.224963+0000","last_active":"2023-11-28T11:17:49.227494+0000","last_peered":"2023-11-28T11:17:49.227494+0000","last_clean":"2023-11-28T11:17:49.227494+0000","last_became_active":"2023-11-28T11:17:49.224679+0000","last_became_peered":"2023-11-28T11:17:49.224679+0000","last_unstale":"2023-11-28T11:17:49.227494+0000","last_undegraded":"2023-11-28T11:17:49.227494+0000","last_fullsized":"2023-11-28T11:17:49.227494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200107+0000","last_change":"2023-11-28T11:17:48.690687+0000","last_active":"2023-11-28T11:17:49.200107+0000","last_peered":"2023-11-28T11:17:49.200107+0000","last_clean":"2023-11-28T11:17:49.200107+0000","last_became_active":"2023-11-28T11:17:48.690509+0000","last_became_peered":"2023-11-28T11:17:48.690509+0000","last_unstale":"2023-11-28T11:17:49.200107+0000","last_undegraded":"2023-11-28T11:17:49.200107+0000","last_fullsized":"2023-11-28T11:17:49.200107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.217149+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:49.217149+0000","last_peered":"2023-11-28T11:17:49.217149+0000","last_clean":"2023-11-28T11:17:49.217149+0000","last_became_active":"2023-11-28T11:17:49.215446+0000","last_became_peered":"2023-11-28T11:17:49.215446+0000","last_unstale":"2023-11-28T11:17:49.217149+0000","last_undegraded":"2023-11-28T11:17:49.217149+0000","last_fullsized":"2023-11-28T11:17:49.217149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.206127+0000","last_change":"2023-11-28T11:17:48.689522+0000","last_active":"2023-11-28T11:17:49.206127+0000","last_peered":"2023-11-28T11:17:49.206127+0000","last_clean":"2023-11-28T11:17:49.206127+0000","last_became_active":"2023-11-28T11:17:48.689211+0000","last_became_peered":"2023-11-28T11:17:48.689211+0000","last_unstale":"2023-11-28T11:17:49.206127+0000","last_undegraded":"2023-11-28T11:17:49.206127+0000","last_fullsized":"2023-11-28T11:17:49.206127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225099+0000","last_change":"2023-11-28T11:17:49.222548+0000","last_active":"2023-11-28T11:17:49.225099+0000","last_peered":"2023-11-28T11:17:49.225099+0000","last_clean":"2023-11-28T11:17:49.225099+0000","last_became_active":"2023-11-28T11:17:49.222341+0000","last_became_peered":"2023-11-28T11:17:49.222341+0000","last_unstale":"2023-11-28T11:17:49.225099+0000","last_undegraded":"2023-11-28T11:17:49.225099+0000","last_fullsized":"2023-11-28T11:17:49.225099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199728+0000","last_change":"2023-11-28T11:17:48.696385+0000","last_active":"2023-11-28T11:17:50.199728+0000","last_peered":"2023-11-28T11:17:50.199728+0000","last_clean":"2023-11-28T11:17:50.199728+0000","last_became_active":"2023-11-28T11:17:48.696028+0000","last_became_peered":"2023-11-28T11:17:48.696028+0000","last_unstale":"2023-11-28T11:17:50.199728+0000","last_undegraded":"2023-11-28T11:17:50.199728+0000","last_fullsized":"2023-11-28T11:17:50.199728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:49.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.214215+0000","last_change":"2023-11-28T11:17:49.210958+0000","last_active":"2023-11-28T11:17:49.214215+0000","last_peered":"2023-11-28T11:17:49.214215+0000","last_clean":"2023-11-28T11:17:49.214215+0000","last_became_active":"2023-11-28T11:17:49.210651+0000","last_became_peered":"2023-11-28T11:17:49.210651+0000","last_unstale":"2023-11-28T11:17:49.214215+0000","last_undegraded":"2023-11-28T11:17:49.214215+0000","last_fullsized":"2023-11-28T11:17:49.214215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224370+0000","last_change":"2023-11-28T11:17:48.697605+0000","last_active":"2023-11-28T11:17:49.224370+0000","last_peered":"2023-11-28T11:17:49.224370+0000","last_clean":"2023-11-28T11:17:49.224370+0000","last_became_active":"2023-11-28T11:17:48.696308+0000","last_became_peered":"2023-11-28T11:17:48.696308+0000","last_unstale":"2023-11-28T11:17:49.224370+0000","last_undegraded":"2023-11-28T11:17:49.224370+0000","last_fullsized":"2023-11-28T11:17:49.224370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:46.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198244+0000","last_change":"2023-11-28T11:17:49.213923+0000","last_active":"2023-11-28T11:17:50.198244+0000","last_peered":"2023-11-28T11:17:50.198244+0000","last_clean":"2023-11-28T11:17:50.198244+0000","last_became_active":"2023-11-28T11:17:49.213693+0000","last_became_peered":"2023-11-28T11:17:49.213693+0000","last_unstale":"2023-11-28T11:17:50.198244+0000","last_undegraded":"2023-11-28T11:17:50.198244+0000","last_fullsized":"2023-11-28T11:17:50.198244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200624+0000","last_change":"2023-11-28T11:17:48.693861+0000","last_active":"2023-11-28T11:17:50.200624+0000","last_peered":"2023-11-28T11:17:50.200624+0000","last_clean":"2023-11-28T11:17:50.200624+0000","last_became_active":"2023-11-28T11:17:48.693491+0000","last_became_peered":"2023-11-28T11:17:48.693491+0000","last_unstale":"2023-11-28T11:17:50.200624+0000","last_undegraded":"2023-11-28T11:17:50.200624+0000","last_fullsized":"2023-11-28T11:17:50.200624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200248+0000","last_change":"2023-11-28T11:17:49.213236+0000","last_active":"2023-11-28T11:17:50.200248+0000","last_peered":"2023-11-28T11:17:50.200248+0000","last_clean":"2023-11-28T11:17:50.200248+0000","last_became_active":"2023-11-28T11:17:49.211752+0000","last_became_peered":"2023-11-28T11:17:49.211752+0000","last_unstale":"2023-11-28T11:17:50.200248+0000","last_undegraded":"2023-11-28T11:17:50.200248+0000","last_fullsized":"2023-11-28T11:17:50.200248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:40.807651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226610+0000","last_change":"2023-11-28T11:17:48.702529+0000","last_active":"2023-11-28T11:17:49.226610+0000","last_peered":"2023-11-28T11:17:49.226610+0000","last_clean":"2023-11-28T11:17:49.226610+0000","last_became_active":"2023-11-28T11:17:48.702137+0000","last_became_peered":"2023-11-28T11:17:48.702137+0000","last_unstale":"2023-11-28T11:17:49.226610+0000","last_undegraded":"2023-11-28T11:17:49.226610+0000","last_fullsized":"2023-11-28T11:17:49.226610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:49.562684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225145+0000","last_change":"2023-11-28T11:17:49.223453+0000","last_active":"2023-11-28T11:17:49.225145+0000","last_peered":"2023-11-28T11:17:49.225145+0000","last_clean":"2023-11-28T11:17:49.225145+0000","last_became_active":"2023-11-28T11:17:49.222771+0000","last_became_peered":"2023-11-28T11:17:49.222771+0000","last_unstale":"2023-11-28T11:17:49.225145+0000","last_undegraded":"2023-11-28T11:17:49.225145+0000","last_fullsized":"2023-11-28T11:17:49.225145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202421+0000","last_change":"2023-11-28T11:17:48.689983+0000","last_active":"2023-11-28T11:17:49.202421+0000","last_peered":"2023-11-28T11:17:49.202421+0000","last_clean":"2023-11-28T11:17:49.202421+0000","last_became_active":"2023-11-28T11:17:48.688579+0000","last_became_peered":"2023-11-28T11:17:48.688579+0000","last_unstale":"2023-11-28T11:17:49.202421+0000","last_undegraded":"2023-11-28T11:17:49.202421+0000","last_fullsized":"2023-11-28T11:17:49.202421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199828+0000","last_change":"2023-11-28T11:17:49.219334+0000","last_active":"2023-11-28T11:17:50.199828+0000","last_peered":"2023-11-28T11:17:50.199828+0000","last_clean":"2023-11-28T11:17:50.199828+0000","last_became_active":"2023-11-28T11:17:49.218912+0000","last_became_peered":"2023-11-28T11:17:49.218912+0000","last_unstale":"2023-11-28T11:17:50.199828+0000","last_undegraded":"2023-11-28T11:17:50.199828+0000","last_fullsized":"2023-11-28T11:17:50.199828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:29.752626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200248+0000","last_change":"2023-11-28T11:17:48.702427+0000","last_active":"2023-11-28T11:17:49.200248+0000","last_peered":"2023-11-28T11:17:49.200248+0000","last_clean":"2023-11-28T11:17:49.200248+0000","last_became_active":"2023-11-28T11:17:48.702118+0000","last_became_peered":"2023-11-28T11:17:48.702118+0000","last_unstale":"2023-11-28T11:17:49.200248+0000","last_undegraded":"2023-11-28T11:17:49.200248+0000","last_fullsized":"2023-11-28T11:17:49.200248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:37.239541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221826+0000","last_change":"2023-11-28T11:17:49.209958+0000","last_active":"2023-11-28T11:17:49.221826+0000","last_peered":"2023-11-28T11:17:49.221826+0000","last_clean":"2023-11-28T11:17:49.221826+0000","last_became_active":"2023-11-28T11:17:49.209616+0000","last_became_peered":"2023-11-28T11:17:49.209616+0000","last_unstale":"2023-11-28T11:17:49.221826+0000","last_undegraded":"2023-11-28T11:17:49.221826+0000","last_fullsized":"2023-11-28T11:17:49.221826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.231167+0000","last_change":"2023-11-28T11:17:48.696749+0000","last_active":"2023-11-28T11:17:49.231167+0000","last_peered":"2023-11-28T11:17:49.231167+0000","last_clean":"2023-11-28T11:17:49.231167+0000","last_became_active":"2023-11-28T11:17:48.695400+0000","last_became_peered":"2023-11-28T11:17:48.695400+0000","last_unstale":"2023-11-28T11:17:49.231167+0000","last_undegraded":"2023-11-28T11:17:49.231167+0000","last_fullsized":"2023-11-28T11:17:49.231167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.214383+0000","last_change":"2023-11-28T11:17:49.209165+0000","last_active":"2023-11-28T11:17:49.214383+0000","last_peered":"2023-11-28T11:17:49.214383+0000","last_clean":"2023-11-28T11:17:49.214383+0000","last_became_active":"2023-11-28T11:17:49.208916+0000","last_became_peered":"2023-11-28T11:17:49.208916+0000","last_unstale":"2023-11-28T11:17:49.214383+0000","last_undegraded":"2023-11-28T11:17:49.214383+0000","last_fullsized":"2023-11-28T11:17:49.214383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.225371+0000","last_change":"2023-11-28T11:17:48.700934+0000","last_active":"2023-11-28T11:17:49.225371+0000","last_peered":"2023-11-28T11:17:49.225371+0000","last_clean":"2023-11-28T11:17:49.225371+0000","last_became_active":"2023-11-28T11:17:48.700549+0000","last_became_peered":"2023-11-28T11:17:48.700549+0000","last_unstale":"2023-11-28T11:17:49.225371+0000","last_undegraded":"2023-11-28T11:17:49.225371+0000","last_fullsized":"2023-11-28T11:17:49.225371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:40.639240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199331+0000","last_change":"2023-11-28T11:17:49.218909+0000","last_active":"2023-11-28T11:17:50.199331+0000","last_peered":"2023-11-28T11:17:50.199331+0000","last_clean":"2023-11-28T11:17:50.199331+0000","last_became_active":"2023-11-28T11:17:49.218582+0000","last_became_peered":"2023-11-28T11:17:49.218582+0000","last_unstale":"2023-11-28T11:17:50.199331+0000","last_undegraded":"2023-11-28T11:17:50.199331+0000","last_fullsized":"2023-11-28T11:17:50.199331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:25.463820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226887+0000","last_change":"2023-11-28T11:17:48.698986+0000","last_active":"2023-11-28T11:17:49.226887+0000","last_peered":"2023-11-28T11:17:49.226887+0000","last_clean":"2023-11-28T11:17:49.226887+0000","last_became_active":"2023-11-28T11:17:48.698594+0000","last_became_peered":"2023-11-28T11:17:48.698594+0000","last_unstale":"2023-11-28T11:17:49.226887+0000","last_undegraded":"2023-11-28T11:17:49.226887+0000","last_fullsized":"2023-11-28T11:17:49.226887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:19.087105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199992+0000","last_change":"2023-11-28T11:17:49.213092+0000","last_active":"2023-11-28T11:17:50.199992+0000","last_peered":"2023-11-28T11:17:50.199992+0000","last_clean":"2023-11-28T11:17:50.199992+0000","last_became_active":"2023-11-28T11:17:49.211870+0000","last_became_peered":"2023-11-28T11:17:49.211870+0000","last_unstale":"2023-11-28T11:17:50.199992+0000","last_undegraded":"2023-11-28T11:17:50.199992+0000","last_fullsized":"2023-11-28T11:17:50.199992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:41.645456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.228014+0000","last_change":"2023-11-28T11:17:48.695292+0000","last_active":"2023-11-28T11:17:49.228014+0000","last_peered":"2023-11-28T11:17:49.228014+0000","last_clean":"2023-11-28T11:17:49.228014+0000","last_became_active":"2023-11-28T11:17:48.694985+0000","last_became_peered":"2023-11-28T11:17:48.694985+0000","last_unstale":"2023-11-28T11:17:49.228014+0000","last_undegraded":"2023-11-28T11:17:49.228014+0000","last_fullsized":"2023-11-28T11:17:49.228014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:56.077529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227641+0000","last_change":"2023-11-28T11:17:49.226629+0000","last_active":"2023-11-28T11:17:49.227641+0000","last_peered":"2023-11-28T11:17:49.227641+0000","last_clean":"2023-11-28T11:17:49.227641+0000","last_became_active":"2023-11-28T11:17:49.226347+0000","last_became_peered":"2023-11-28T11:17:49.226347+0000","last_unstale":"2023-11-28T11:17:49.227641+0000","last_undegraded":"2023-11-28T11:17:49.227641+0000","last_fullsized":"2023-11-28T11:17:49.227641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199618+0000","last_change":"2023-11-28T11:17:48.692881+0000","last_active":"2023-11-28T11:17:50.199618+0000","last_peered":"2023-11-28T11:17:50.199618+0000","last_clean":"2023-11-28T11:17:50.199618+0000","last_became_active":"2023-11-28T11:17:48.692426+0000","last_became_peered":"2023-11-28T11:17:48.692426+0000","last_unstale":"2023-11-28T11:17:50.199618+0000","last_undegraded":"2023-11-28T11:17:50.199618+0000","last_fullsized":"2023-11-28T11:17:50.199618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:23.247523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.226003+0000","last_change":"2023-11-28T11:17:49.221185+0000","last_active":"2023-11-28T11:17:49.226003+0000","last_peered":"2023-11-28T11:17:49.226003+0000","last_clean":"2023-11-28T11:17:49.226003+0000","last_became_active":"2023-11-28T11:17:49.220927+0000","last_became_peered":"2023-11-28T11:17:49.220927+0000","last_unstale":"2023-11-28T11:17:49.226003+0000","last_undegraded":"2023-11-28T11:17:49.226003+0000","last_fullsized":"2023-11-28T11:17:49.226003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224861+0000","last_change":"2023-11-28T11:17:48.698335+0000","last_active":"2023-11-28T11:17:49.224861+0000","last_peered":"2023-11-28T11:17:49.224861+0000","last_clean":"2023-11-28T11:17:49.224861+0000","last_became_active":"2023-11-28T11:17:48.698032+0000","last_became_peered":"2023-11-28T11:17:48.698032+0000","last_unstale":"2023-11-28T11:17:49.224861+0000","last_undegraded":"2023-11-28T11:17:49.224861+0000","last_fullsized":"2023-11-28T11:17:49.224861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200609+0000","last_change":"2023-11-28T11:17:49.207155+0000","last_active":"2023-11-28T11:17:50.200609+0000","last_peered":"2023-11-28T11:17:50.200609+0000","last_clean":"2023-11-28T11:17:50.200609+0000","last_became_active":"2023-11-28T11:17:49.206800+0000","last_became_peered":"2023-11-28T11:17:49.206800+0000","last_unstale":"2023-11-28T11:17:50.200609+0000","last_undegraded":"2023-11-28T11:17:50.200609+0000","last_fullsized":"2023-11-28T11:17:50.200609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.206646+0000","last_change":"2023-11-28T11:17:48.702363+0000","last_active":"2023-11-28T11:17:49.206646+0000","last_peered":"2023-11-28T11:17:49.206646+0000","last_clean":"2023-11-28T11:17:49.206646+0000","last_became_active":"2023-11-28T11:17:48.701778+0000","last_became_peered":"2023-11-28T11:17:48.701778+0000","last_unstale":"2023-11-28T11:17:49.206646+0000","last_undegraded":"2023-11-28T11:17:49.206646+0000","last_fullsized":"2023-11-28T11:17:49.206646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200788+0000","last_change":"2023-11-28T11:17:49.205845+0000","last_active":"2023-11-28T11:17:50.200788+0000","last_peered":"2023-11-28T11:17:50.200788+0000","last_clean":"2023-11-28T11:17:50.200788+0000","last_became_active":"2023-11-28T11:17:49.205302+0000","last_became_peered":"2023-11-28T11:17:49.205302+0000","last_unstale":"2023-11-28T11:17:50.200788+0000","last_undegraded":"2023-11-28T11:17:50.200788+0000","last_fullsized":"2023-11-28T11:17:50.200788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200199+0000","last_change":"2023-11-28T11:17:48.699968+0000","last_active":"2023-11-28T11:17:50.200199+0000","last_peered":"2023-11-28T11:17:50.200199+0000","last_clean":"2023-11-28T11:17:50.200199+0000","last_became_active":"2023-11-28T11:17:48.697897+0000","last_became_peered":"2023-11-28T11:17:48.697897+0000","last_unstale":"2023-11-28T11:17:50.200199+0000","last_undegraded":"2023-11-28T11:17:50.200199+0000","last_fullsized":"2023-11-28T11:17:50.200199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200758+0000","last_change":"2023-11-28T11:17:49.210939+0000","last_active":"2023-11-28T11:17:50.200758+0000","last_peered":"2023-11-28T11:17:50.200758+0000","last_clean":"2023-11-28T11:17:50.200758+0000","last_became_active":"2023-11-28T11:17:49.210585+0000","last_became_peered":"2023-11-28T11:17:49.210585+0000","last_unstale":"2023-11-28T11:17:50.200758+0000","last_undegraded":"2023-11-28T11:17:50.200758+0000","last_fullsized":"2023-11-28T11:17:50.200758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:17.930855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.200000+0000","last_change":"2023-11-28T11:17:48.687630+0000","last_active":"2023-11-28T11:17:49.200000+0000","last_peered":"2023-11-28T11:17:49.200000+0000","last_clean":"2023-11-28T11:17:49.200000+0000","last_became_active":"2023-11-28T11:17:48.685494+0000","last_became_peered":"2023-11-28T11:17:48.685494+0000","last_unstale":"2023-11-28T11:17:49.200000+0000","last_undegraded":"2023-11-28T11:17:49.200000+0000","last_fullsized":"2023-11-28T11:17:49.200000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198866+0000","last_change":"2023-11-28T11:17:49.223631+0000","last_active":"2023-11-28T11:17:50.198866+0000","last_peered":"2023-11-28T11:17:50.198866+0000","last_clean":"2023-11-28T11:17:50.198866+0000","last_became_active":"2023-11-28T11:17:49.223321+0000","last_became_peered":"2023-11-28T11:17:49.223321+0000","last_unstale":"2023-11-28T11:17:50.198866+0000","last_undegraded":"2023-11-28T11:17:50.198866+0000","last_fullsized":"2023-11-28T11:17:50.198866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.195384+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:17:49.195384+0000","last_peered":"2023-11-28T11:17:49.195384+0000","last_clean":"2023-11-28T11:17:49.195384+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:17:49.195384+0000","last_undegraded":"2023-11-28T11:17:49.195384+0000","last_fullsized":"2023-11-28T11:17:49.195384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199900+0000","last_change":"2023-11-28T11:17:48.691483+0000","last_active":"2023-11-28T11:17:50.199900+0000","last_peered":"2023-11-28T11:17:50.199900+0000","last_clean":"2023-11-28T11:17:50.199900+0000","last_became_active":"2023-11-28T11:17:48.691194+0000","last_became_peered":"2023-11-28T11:17:48.691194+0000","last_unstale":"2023-11-28T11:17:50.199900+0000","last_undegraded":"2023-11-28T11:17:50.199900+0000","last_fullsized":"2023-11-28T11:17:50.199900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:43.938903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198294+0000","last_change":"2023-11-28T11:17:49.217104+0000","last_active":"2023-11-28T11:17:50.198294+0000","last_peered":"2023-11-28T11:17:50.198294+0000","last_clean":"2023-11-28T11:17:50.198294+0000","last_became_active":"2023-11-28T11:17:49.216883+0000","last_became_peered":"2023-11-28T11:17:49.216883+0000","last_unstale":"2023-11-28T11:17:50.198294+0000","last_undegraded":"2023-11-28T11:17:50.198294+0000","last_fullsized":"2023-11-28T11:17:50.198294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202595+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:17:49.202595+0000","last_peered":"2023-11-28T11:17:49.202595+0000","last_clean":"2023-11-28T11:17:49.202595+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:17:49.202595+0000","last_undegraded":"2023-11-28T11:17:49.202595+0000","last_fullsized":"2023-11-28T11:17:49.202595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200374+0000","last_change":"2023-11-28T11:17:48.699726+0000","last_active":"2023-11-28T11:17:50.200374+0000","last_peered":"2023-11-28T11:17:50.200374+0000","last_clean":"2023-11-28T11:17:50.200374+0000","last_became_active":"2023-11-28T11:17:48.699409+0000","last_became_peered":"2023-11-28T11:17:48.699409+0000","last_unstale":"2023-11-28T11:17:50.200374+0000","last_undegraded":"2023-11-28T11:17:50.200374+0000","last_fullsized":"2023-11-28T11:17:50.200374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198558+0000","last_change":"2023-11-28T11:17:49.223120+0000","last_active":"2023-11-28T11:17:50.198558+0000","last_peered":"2023-11-28T11:17:50.198558+0000","last_clean":"2023-11-28T11:17:50.198558+0000","last_became_active":"2023-11-28T11:17:49.217684+0000","last_became_peered":"2023-11-28T11:17:49.217684+0000","last_unstale":"2023-11-28T11:17:50.198558+0000","last_undegraded":"2023-11-28T11:17:50.198558+0000","last_fullsized":"2023-11-28T11:17:50.198558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.694120+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:17:48.694120+0000","last_peered":"2023-11-28T11:17:48.694120+0000","last_clean":"2023-11-28T11:17:48.694120+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:17:48.694120+0000","last_undegraded":"2023-11-28T11:17:48.694120+0000","last_fullsized":"2023-11-28T11:17:48.694120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198094+0000","last_change":"2023-11-28T11:17:48.695369+0000","last_active":"2023-11-28T11:17:50.198094+0000","last_peered":"2023-11-28T11:17:50.198094+0000","last_clean":"2023-11-28T11:17:50.198094+0000","last_became_active":"2023-11-28T11:17:48.695065+0000","last_became_peered":"2023-11-28T11:17:48.695065+0000","last_unstale":"2023-11-28T11:17:50.198094+0000","last_undegraded":"2023-11-28T11:17:50.198094+0000","last_fullsized":"2023-11-28T11:17:50.198094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198772+0000","last_change":"2023-11-28T11:17:49.217116+0000","last_active":"2023-11-28T11:17:50.198772+0000","last_peered":"2023-11-28T11:17:50.198772+0000","last_clean":"2023-11-28T11:17:50.198772+0000","last_became_active":"2023-11-28T11:17:49.216818+0000","last_became_peered":"2023-11-28T11:17:49.216818+0000","last_unstale":"2023-11-28T11:17:50.198772+0000","last_undegraded":"2023-11-28T11:17:50.198772+0000","last_fullsized":"2023-11-28T11:17:50.198772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202500+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:17:49.202500+0000","last_peered":"2023-11-28T11:17:49.202500+0000","last_clean":"2023-11-28T11:17:49.202500+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:17:49.202500+0000","last_undegraded":"2023-11-28T11:17:49.202500+0000","last_fullsized":"2023-11-28T11:17:49.202500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.686831+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:17:48.686831+0000","last_peered":"2023-11-28T11:17:48.686831+0000","last_clean":"2023-11-28T11:17:48.686831+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:17:48.686831+0000","last_undegraded":"2023-11-28T11:17:48.686831+0000","last_fullsized":"2023-11-28T11:17:48.686831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.197354+0000","last_change":"2023-11-28T11:17:48.692995+0000","last_active":"2023-11-28T11:17:49.197354+0000","last_peered":"2023-11-28T11:17:49.197354+0000","last_clean":"2023-11-28T11:17:49.197354+0000","last_became_active":"2023-11-28T11:17:48.692619+0000","last_became_peered":"2023-11-28T11:17:48.692619+0000","last_unstale":"2023-11-28T11:17:49.197354+0000","last_undegraded":"2023-11-28T11:17:49.197354+0000","last_fullsized":"2023-11-28T11:17:49.197354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212711+0000","last_change":"2023-11-28T11:17:49.211592+0000","last_active":"2023-11-28T11:17:49.212711+0000","last_peered":"2023-11-28T11:17:49.212711+0000","last_clean":"2023-11-28T11:17:49.212711+0000","last_became_active":"2023-11-28T11:17:49.209606+0000","last_became_peered":"2023-11-28T11:17:49.209606+0000","last_unstale":"2023-11-28T11:17:49.212711+0000","last_undegraded":"2023-11-28T11:17:49.212711+0000","last_fullsized":"2023-11-28T11:17:49.212711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:47.444676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.687012+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:17:48.687012+0000","last_peered":"2023-11-28T11:17:48.687012+0000","last_clean":"2023-11-28T11:17:48.687012+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:17:48.687012+0000","last_undegraded":"2023-11-28T11:17:48.687012+0000","last_fullsized":"2023-11-28T11:17:48.687012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202413+0000","last_change":"2023-11-28T11:17:48.696786+0000","last_active":"2023-11-28T11:17:49.202413+0000","last_peered":"2023-11-28T11:17:49.202413+0000","last_clean":"2023-11-28T11:17:49.202413+0000","last_became_active":"2023-11-28T11:17:48.696474+0000","last_became_peered":"2023-11-28T11:17:48.696474+0000","last_unstale":"2023-11-28T11:17:49.202413+0000","last_undegraded":"2023-11-28T11:17:49.202413+0000","last_fullsized":"2023-11-28T11:17:49.202413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227476+0000","last_change":"2023-11-28T11:17:49.219523+0000","last_active":"2023-11-28T11:17:49.227476+0000","last_peered":"2023-11-28T11:17:49.227476+0000","last_clean":"2023-11-28T11:17:49.227476+0000","last_became_active":"2023-11-28T11:17:49.219226+0000","last_became_peered":"2023-11-28T11:17:49.219226+0000","last_unstale":"2023-11-28T11:17:49.227476+0000","last_undegraded":"2023-11-28T11:17:49.227476+0000","last_fullsized":"2023-11-28T11:17:49.227476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.688233+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:17:48.688233+0000","last_peered":"2023-11-28T11:17:48.688233+0000","last_clean":"2023-11-28T11:17:48.688233+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:17:48.688233+0000","last_undegraded":"2023-11-28T11:17:48.688233+0000","last_fullsized":"2023-11-28T11:17:48.688233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198637+0000","last_change":"2023-11-28T11:17:48.696130+0000","last_active":"2023-11-28T11:17:50.198637+0000","last_peered":"2023-11-28T11:17:50.198637+0000","last_clean":"2023-11-28T11:17:50.198637+0000","last_became_active":"2023-11-28T11:17:48.695765+0000","last_became_peered":"2023-11-28T11:17:48.695765+0000","last_unstale":"2023-11-28T11:17:50.198637+0000","last_undegraded":"2023-11-28T11:17:50.198637+0000","last_fullsized":"2023-11-28T11:17:50.198637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223314+0000","last_change":"2023-11-28T11:17:49.219689+0000","last_active":"2023-11-28T11:17:49.223314+0000","last_peered":"2023-11-28T11:17:49.223314+0000","last_clean":"2023-11-28T11:17:49.223314+0000","last_became_active":"2023-11-28T11:17:49.219446+0000","last_became_peered":"2023-11-28T11:17:49.219446+0000","last_unstale":"2023-11-28T11:17:49.223314+0000","last_undegraded":"2023-11-28T11:17:49.223314+0000","last_fullsized":"2023-11-28T11:17:49.223314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224790+0000","last_change":"2023-11-28T11:17:48.694653+0000","last_active":"2023-11-28T11:17:49.224790+0000","last_peered":"2023-11-28T11:17:49.224790+0000","last_clean":"2023-11-28T11:17:49.224790+0000","last_became_active":"2023-11-28T11:17:48.694360+0000","last_became_peered":"2023-11-28T11:17:48.694360+0000","last_unstale":"2023-11-28T11:17:49.224790+0000","last_undegraded":"2023-11-28T11:17:49.224790+0000","last_fullsized":"2023-11-28T11:17:49.224790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.194295+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:17:49.194295+0000","last_peered":"2023-11-28T11:17:49.194295+0000","last_clean":"2023-11-28T11:17:49.194295+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:17:49.194295+0000","last_undegraded":"2023-11-28T11:17:49.194295+0000","last_fullsized":"2023-11-28T11:17:49.194295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.223458+0000","last_change":"2023-11-28T11:17:49.219150+0000","last_active":"2023-11-28T11:17:49.223458+0000","last_peered":"2023-11-28T11:17:49.223458+0000","last_clean":"2023-11-28T11:17:49.223458+0000","last_became_active":"2023-11-28T11:17:49.218887+0000","last_became_peered":"2023-11-28T11:17:49.218887+0000","last_unstale":"2023-11-28T11:17:49.223458+0000","last_undegraded":"2023-11-28T11:17:49.223458+0000","last_fullsized":"2023-11-28T11:17:49.223458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:17:48.686699+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:17:48.686699+0000","last_peered":"2023-11-28T11:17:48.686699+0000","last_clean":"2023-11-28T11:17:48.686699+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:17:48.686699+0000","last_undegraded":"2023-11-28T11:17:48.686699+0000","last_fullsized":"2023-11-28T11:17:48.686699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.207435+0000","last_change":"2023-11-28T11:17:48.688812+0000","last_active":"2023-11-28T11:17:49.207435+0000","last_peered":"2023-11-28T11:17:49.207435+0000","last_clean":"2023-11-28T11:17:49.207435+0000","last_became_active":"2023-11-28T11:17:48.688551+0000","last_became_peered":"2023-11-28T11:17:48.688551+0000","last_unstale":"2023-11-28T11:17:49.207435+0000","last_undegraded":"2023-11-28T11:17:49.207435+0000","last_fullsized":"2023-11-28T11:17:49.207435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200103+0000","last_change":"2023-11-28T11:17:49.207524+0000","last_active":"2023-11-28T11:17:50.200103+0000","last_peered":"2023-11-28T11:17:50.200103+0000","last_clean":"2023-11-28T11:17:50.200103+0000","last_became_active":"2023-11-28T11:17:49.207175+0000","last_became_peered":"2023-11-28T11:17:49.207175+0000","last_unstale":"2023-11-28T11:17:50.200103+0000","last_undegraded":"2023-11-28T11:17:50.200103+0000","last_fullsized":"2023-11-28T11:17:50.200103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:42.481462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227392+0000","last_change":"2023-11-28T11:17:48.691892+0000","last_active":"2023-11-28T11:17:49.227392+0000","last_peered":"2023-11-28T11:17:49.227392+0000","last_clean":"2023-11-28T11:17:49.227392+0000","last_became_active":"2023-11-28T11:17:48.691619+0000","last_became_peered":"2023-11-28T11:17:48.691619+0000","last_unstale":"2023-11-28T11:17:49.227392+0000","last_undegraded":"2023-11-28T11:17:49.227392+0000","last_fullsized":"2023-11-28T11:17:49.227392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.230934+0000","last_change":"2023-11-28T11:17:49.230012+0000","last_active":"2023-11-28T11:17:49.230934+0000","last_peered":"2023-11-28T11:17:49.230934+0000","last_clean":"2023-11-28T11:17:49.230934+0000","last_became_active":"2023-11-28T11:17:49.229696+0000","last_became_peered":"2023-11-28T11:17:49.229696+0000","last_unstale":"2023-11-28T11:17:49.230934+0000","last_undegraded":"2023-11-28T11:17:49.230934+0000","last_fullsized":"2023-11-28T11:17:49.230934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199712+0000","last_change":"2023-11-28T11:17:48.699672+0000","last_active":"2023-11-28T11:17:50.199712+0000","last_peered":"2023-11-28T11:17:50.199712+0000","last_clean":"2023-11-28T11:17:50.199712+0000","last_became_active":"2023-11-28T11:17:48.699372+0000","last_became_peered":"2023-11-28T11:17:48.699372+0000","last_unstale":"2023-11-28T11:17:50.199712+0000","last_undegraded":"2023-11-28T11:17:50.199712+0000","last_fullsized":"2023-11-28T11:17:50.199712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:53.370355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200423+0000","last_change":"2023-11-28T11:17:49.212871+0000","last_active":"2023-11-28T11:17:50.200423+0000","last_peered":"2023-11-28T11:17:50.200423+0000","last_clean":"2023-11-28T11:17:50.200423+0000","last_became_active":"2023-11-28T11:17:49.212601+0000","last_became_peered":"2023-11-28T11:17:49.212601+0000","last_unstale":"2023-11-28T11:17:50.200423+0000","last_undegraded":"2023-11-28T11:17:50.200423+0000","last_fullsized":"2023-11-28T11:17:50.200423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199934+0000","last_change":"2023-11-28T11:17:48.697131+0000","last_active":"2023-11-28T11:17:50.199934+0000","last_peered":"2023-11-28T11:17:50.199934+0000","last_clean":"2023-11-28T11:17:50.199934+0000","last_became_active":"2023-11-28T11:17:48.696849+0000","last_became_peered":"2023-11-28T11:17:48.696849+0000","last_unstale":"2023-11-28T11:17:50.199934+0000","last_undegraded":"2023-11-28T11:17:50.199934+0000","last_fullsized":"2023-11-28T11:17:50.199934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:09.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200297+0000","last_change":"2023-11-28T11:17:49.216478+0000","last_active":"2023-11-28T11:17:50.200297+0000","last_peered":"2023-11-28T11:17:50.200297+0000","last_clean":"2023-11-28T11:17:50.200297+0000","last_became_active":"2023-11-28T11:17:49.215792+0000","last_became_peered":"2023-11-28T11:17:49.215792+0000","last_unstale":"2023-11-28T11:17:50.200297+0000","last_undegraded":"2023-11-28T11:17:50.200297+0000","last_fullsized":"2023-11-28T11:17:50.200297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.220326+0000","last_change":"2023-11-28T11:17:48.692669+0000","last_active":"2023-11-28T11:17:49.220326+0000","last_peered":"2023-11-28T11:17:49.220326+0000","last_clean":"2023-11-28T11:17:49.220326+0000","last_became_active":"2023-11-28T11:17:48.691780+0000","last_became_peered":"2023-11-28T11:17:48.691780+0000","last_unstale":"2023-11-28T11:17:49.220326+0000","last_undegraded":"2023-11-28T11:17:49.220326+0000","last_fullsized":"2023-11-28T11:17:49.220326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.222953+0000","last_change":"2023-11-28T11:17:49.220031+0000","last_active":"2023-11-28T11:17:49.222953+0000","last_peered":"2023-11-28T11:17:49.222953+0000","last_clean":"2023-11-28T11:17:49.222953+0000","last_became_active":"2023-11-28T11:17:49.219784+0000","last_became_peered":"2023-11-28T11:17:49.219784+0000","last_unstale":"2023-11-28T11:17:49.222953+0000","last_undegraded":"2023-11-28T11:17:49.222953+0000","last_fullsized":"2023-11-28T11:17:49.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.227804+0000","last_change":"2023-11-28T11:17:48.689149+0000","last_active":"2023-11-28T11:17:49.227804+0000","last_peered":"2023-11-28T11:17:49.227804+0000","last_clean":"2023-11-28T11:17:49.227804+0000","last_became_active":"2023-11-28T11:17:48.688826+0000","last_became_peered":"2023-11-28T11:17:48.688826+0000","last_unstale":"2023-11-28T11:17:49.227804+0000","last_undegraded":"2023-11-28T11:17:49.227804+0000","last_fullsized":"2023-11-28T11:17:49.227804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199319+0000","last_change":"2023-11-28T11:17:49.208765+0000","last_active":"2023-11-28T11:17:50.199319+0000","last_peered":"2023-11-28T11:17:50.199319+0000","last_clean":"2023-11-28T11:17:50.199319+0000","last_became_active":"2023-11-28T11:17:49.208432+0000","last_became_peered":"2023-11-28T11:17:49.208432+0000","last_unstale":"2023-11-28T11:17:50.199319+0000","last_undegraded":"2023-11-28T11:17:50.199319+0000","last_fullsized":"2023-11-28T11:17:50.199319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.212723+0000","last_change":"2023-11-28T11:17:49.210834+0000","last_active":"2023-11-28T11:17:49.212723+0000","last_peered":"2023-11-28T11:17:49.212723+0000","last_clean":"2023-11-28T11:17:49.212723+0000","last_became_active":"2023-11-28T11:17:49.208613+0000","last_became_peered":"2023-11-28T11:17:49.208613+0000","last_unstale":"2023-11-28T11:17:49.212723+0000","last_undegraded":"2023-11-28T11:17:49.212723+0000","last_fullsized":"2023-11-28T11:17:49.212723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.224638+0000","last_change":"2023-11-28T11:17:49.223484+0000","last_active":"2023-11-28T11:17:49.224638+0000","last_peered":"2023-11-28T11:17:49.224638+0000","last_clean":"2023-11-28T11:17:49.224638+0000","last_became_active":"2023-11-28T11:17:49.222011+0000","last_became_peered":"2023-11-28T11:17:49.222011+0000","last_unstale":"2023-11-28T11:17:49.224638+0000","last_undegraded":"2023-11-28T11:17:49.224638+0000","last_fullsized":"2023-11-28T11:17:49.224638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200166+0000","last_change":"2023-11-28T11:17:49.216669+0000","last_active":"2023-11-28T11:17:50.200166+0000","last_peered":"2023-11-28T11:17:50.200166+0000","last_clean":"2023-11-28T11:17:50.200166+0000","last_became_active":"2023-11-28T11:17:49.216225+0000","last_became_peered":"2023-11-28T11:17:49.216225+0000","last_unstale":"2023-11-28T11:17:50.200166+0000","last_undegraded":"2023-11-28T11:17:50.200166+0000","last_fullsized":"2023-11-28T11:17:50.200166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:22.361832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.221248+0000","last_change":"2023-11-28T11:17:49.219093+0000","last_active":"2023-11-28T11:17:49.221248+0000","last_peered":"2023-11-28T11:17:49.221248+0000","last_clean":"2023-11-28T11:17:49.221248+0000","last_became_active":"2023-11-28T11:17:49.218818+0000","last_became_peered":"2023-11-28T11:17:49.218818+0000","last_unstale":"2023-11-28T11:17:49.221248+0000","last_undegraded":"2023-11-28T11:17:49.221248+0000","last_fullsized":"2023-11-28T11:17:49.221248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198448+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.198448+0000","last_peered":"2023-11-28T11:17:50.198448+0000","last_clean":"2023-11-28T11:17:50.198448+0000","last_became_active":"2023-11-28T11:17:49.215424+0000","last_became_peered":"2023-11-28T11:17:49.215424+0000","last_unstale":"2023-11-28T11:17:50.198448+0000","last_undegraded":"2023-11-28T11:17:50.198448+0000","last_fullsized":"2023-11-28T11:17:50.198448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13176,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358664,"statfs":{"total":96636764160,"available":96623271936,"internally_reserved":0,"allocated":835584,"data_stored":668259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11992,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359848,"statfs":{"total":96636764160,"available":96624484352,"internally_reserved":0,"allocated":212992,"data_stored":76127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":81816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13148,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358692,"statfs":{"total":96636764160,"available":96623300608,"internally_reserved":0,"allocated":806912,"data_stored":666042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11992,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359848,"statfs":{"total":96636764160,"available":96624484352,"internally_reserved":0,"allocated":212992,"data_stored":76127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":76217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12140,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359700,"statfs":{"total":96636764160,"available":96624332800,"internally_reserved":0,"allocated":233472,"data_stored":77731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12076,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359764,"statfs":{"total":96636764160,"available":96624398336,"internally_reserved":0,"allocated":233472,"data_stored":83416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:17:54.578 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:17:54.579 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:17:54.579 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:17:54.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:17:54.579 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:17:54.660 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-11-28T11:17:54.659+0000 7f6a025f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:17:54.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.816+0000 7f880edff700 1 Processor -- start 2023-11-28T11:17:54.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.816+0000 7f880edff700 1 -- start start 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880edff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8808161340 0x7f88080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880edff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88080d2320 0x7f88080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880edff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88080d1950 con 0x7f880815c9b0 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880edff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88080b5800 con 0x7f8808161340 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880edff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88080b5300 con 0x7f88080d2320 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f8807fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8808161340 0x7f88080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f8807fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8808161340 0x7f88080d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39860/0 (socket says 172.21.15.53:39860) 2023-11-28T11:17:54.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 -- 172.21.15.53:0/3720395512 learned_addr learned my addr 172.21.15.53:0/3720395512 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.817+0000 7f880cb9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f880815c9b0 0x7f88080b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46850/0 (socket says 172.21.15.53:46850) 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f880d39c700 1 --2- 172.21.15.53:0/3720395512 >> v2:172.21.15.157:3300/0 conn(0x7f88080d2320 0x7f88080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 -- 172.21.15.53:0/3720395512 >> v2:172.21.15.157:3300/0 conn(0x7f88080d2320 msgr2=0x7f88080cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 --2- 172.21.15.53:0/3720395512 >> v2:172.21.15.157:3300/0 conn(0x7f88080d2320 0x7f88080cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 -- 172.21.15.53:0/3720395512 >> v1:172.21.15.53:6789/0 conn(0x7f880815c9b0 legacy=0x7f88080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 -- 172.21.15.53:0/3720395512 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080b5520 con 0x7f8808161340 2023-11-28T11:17:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f8807fff700 1 --2- 172.21.15.53:0/3720395512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8808161340 0x7f88080d1de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f880806c170 tx=0x7f87f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90f3a36e96074 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f88077fe700 1 -- 172.21.15.53:0/3720395512 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400b860 con 0x7f8808161340 2023-11-28T11:17:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f88077fe700 1 -- 172.21.15.53:0/3720395512 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f400b9c0 con 0x7f8808161340 2023-11-28T11:17:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.818+0000 7f88077fe700 1 -- 172.21.15.53:0/3720395512 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400bd50 con 0x7f8808161340 2023-11-28T11:17:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 -- 172.21.15.53:0/3720395512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8808161340 msgr2=0x7f88080d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 --2- 172.21.15.53:0/3720395512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8808161340 0x7f88080d1de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f880806c170 tx=0x7f87f4009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 -- 172.21.15.53:0/3720395512 shutdown_connections 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 --2- 172.21.15.53:0/3720395512 >> v2:172.21.15.157:3300/0 conn(0x7f88080d2320 0x7f88080cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 --2- 172.21.15.53:0/3720395512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8808161340 0x7f88080d1de0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 -- 172.21.15.53:0/3720395512 >> 172.21.15.53:0/3720395512 conn(0x7f8808072600 msgr2=0x7f88080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 -- 172.21.15.53:0/3720395512 shutdown_connections 2023-11-28T11:17:54.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.819+0000 7f880edff700 1 -- 172.21.15.53:0/3720395512 wait complete. 2023-11-28T11:17:54.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.821+0000 7f880edff700 1 Processor -- start 2023-11-28T11:17:54.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.821+0000 7f880edff700 1 -- start start 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88080d2320 0x7f8808150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 0x7f8808156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081547e0 con 0x7f880815c9b0 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081e6460 con 0x7f88080d2320 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88081e72c0 con 0x7f8808161340 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f8807fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f8807fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39864/0 (socket says 172.21.15.53:39864) 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f8807fff700 1 -- 172.21.15.53:0/3469616357 learned_addr learned my addr 172.21.15.53:0/3469616357 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880cb9b700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88080d2320 0x7f8808150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.822+0000 7f880d39c700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 0x7f8808156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 -- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 msgr2=0x7f8808156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 0x7f8808156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 -- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88080d2320 msgr2=0x7f8808150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88080d2320 0x7f8808150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 -- 172.21.15.53:0/3469616357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009550 con 0x7f880815c9b0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f880d39c700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 0x7f8808156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f8807fff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87f4000c00 tx=0x7f87f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004380 con 0x7f880815c9b0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f401f070 con 0x7f880815c9b0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40036a0 con 0x7f880815c9b0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88080d15e0 con 0x7f880815c9b0 2023-11-28T11:17:54.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.823+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88080d1300 con 0x7f880815c9b0 2023-11-28T11:17:54.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.824+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880815cd80 con 0x7f880815c9b0 2023-11-28T11:17:54.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.826+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87f402a020 con 0x7f880815c9b0 2023-11-28T11:17:54.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.827+0000 7f88057fa700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 0x7f87ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:54.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.827+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87f4087520 con 0x7f880815c9b0 2023-11-28T11:17:54.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.828+0000 7f880cb9b700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 0x7f87ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:54.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.828+0000 7f880cb9b700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 0x7f87ec06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000f60 tx=0x7f87fc007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:54.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:54.829+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87f4053e20 con 0x7f880815c9b0 2023-11-28T11:17:55.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.087+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f880815d0c0 con 0x7f880815c9b0 2023-11-28T11:17:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.090+0000 7f88057fa700 1 -- 172.21.15.53:0/3469616357 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166384 (secure 0 0 0) 0x7f87f4057440 con 0x7f880815c9b0 2023-11-28T11:17:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 msgr2=0x7f87ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 0x7f87ec06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000f60 tx=0x7f87fc007ed0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 msgr2=0x7f88081e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87f4000c00 tx=0x7f87f4004750 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 shutdown_connections 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87ec0682f0 0x7f87ec06a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8808161340 0x7f8808156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88080d2320 0x7f8808150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 --2- 172.21.15.53:0/3469616357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880815c9b0 0x7f88081e7750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.093+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 >> 172.21.15.53:0/3469616357 conn(0x7f8808072600 msgr2=0x7f88080b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.094+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 shutdown_connections 2023-11-28T11:17:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.094+0000 7f880edff700 1 -- 172.21.15.53:0/3469616357 wait complete. 2023-11-28T11:17:55.112 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":132254},{"type":"v1","addr":"172.21.15.157:6833","nonce":132254}]},"active_addr":"172.21.15.157:6833/132254","active_change":"2023-11-28T11:17:34.919291+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":726348270}]},{"addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3288005821}]},{"addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3598045251}]},{"addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":894521102}]}]} 2023-11-28T11:17:55.116 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:17:55.116 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:17:55.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:55.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.316+0000 7f4435dff700 1 Processor -- start 2023-11-28T11:17:55.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.316+0000 7f4435dff700 1 -- start start 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f4435dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44300d01b0 0x7f44300d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f4435dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f44300b16d0 0x7f44300d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f4435dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44300b0230 con 0x7f44300b20b0 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f4435dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443015cec0 con 0x7f44300b16d0 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f4435dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300b4570 con 0x7f44300d01b0 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f44300b16d0 0x7f44300d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f44300b16d0 0x7f44300d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39872/0 (socket says 172.21.15.53:39872) 2023-11-28T11:17:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 -- 172.21.15.53:0/4119798725 learned_addr learned my addr 172.21.15.53:0/4119798725 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442f7fe700 1 --2- 172.21.15.53:0/4119798725 >> v2:172.21.15.157:3300/0 conn(0x7f44300d01b0 0x7f44300d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 -- 172.21.15.53:0/4119798725 >> v2:172.21.15.157:3300/0 conn(0x7f44300d01b0 msgr2=0x7f44300d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 --2- 172.21.15.53:0/4119798725 >> v2:172.21.15.157:3300/0 conn(0x7f44300d01b0 0x7f44300d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 -- 172.21.15.53:0/4119798725 >> v1:172.21.15.157:6789/0 conn(0x7f44300b20b0 legacy=0x7f44300b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.318+0000 7f442ffff700 1 -- 172.21.15.53:0/4119798725 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300b4850 con 0x7f44300b16d0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f442ffff700 1 --2- 172.21.15.53:0/4119798725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300b16d0 0x7f44300d1160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44300b5bb0 tx=0x7f442800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43421702430afff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f442e7fc700 1 -- 172.21.15.53:0/4119798725 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800e070 con 0x7f44300b16d0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f442e7fc700 1 -- 172.21.15.53:0/4119798725 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442800b910 con 0x7f44300b16d0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f442e7fc700 1 -- 172.21.15.53:0/4119798725 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800bcc0 con 0x7f44300b16d0 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f4435dff700 1 -- 172.21.15.53:0/4119798725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300b16d0 msgr2=0x7f44300d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.319+0000 7f4435dff700 1 --2- 172.21.15.53:0/4119798725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300b16d0 0x7f44300d1160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44300b5bb0 tx=0x7f442800b330 comp rx=0 tx=0).stop 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 -- 172.21.15.53:0/4119798725 shutdown_connections 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 --2- 172.21.15.53:0/4119798725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300b16d0 0x7f44300d1160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 --2- 172.21.15.53:0/4119798725 >> v2:172.21.15.157:3300/0 conn(0x7f44300d01b0 0x7f44300d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 -- 172.21.15.53:0/4119798725 >> 172.21.15.53:0/4119798725 conn(0x7f443009c3f0 msgr2=0x7f44300d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 -- 172.21.15.53:0/4119798725 shutdown_connections 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.320+0000 7f4435dff700 1 -- 172.21.15.53:0/4119798725 wait complete. 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.321+0000 7f4435dff700 1 Processor -- start 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 -- start start 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300b16d0 0x7f4430157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300b20b0 0x7f44301f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4430158f50 con 0x7f44300d01b0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44301f5f80 con 0x7f44300b20b0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.322+0000 7f4435dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4430156110 con 0x7f44300b16d0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39888/0 (socket says 172.21.15.53:39888) 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 -- 172.21.15.53:0/1536516236 learned_addr learned my addr 172.21.15.53:0/1536516236 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442effd700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300b20b0 0x7f44301f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442f7fe700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300b16d0 0x7f4430157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 -- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300b16d0 msgr2=0x7f4430157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300b16d0 0x7f4430157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 -- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300b20b0 msgr2=0x7f44301f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300b20b0 0x7f44301f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 -- 172.21.15.53:0/1536516236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009b30 con 0x7f44300d01b0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.323+0000 7f442ffff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f442800b370 tx=0x7f4428004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.324+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428004880 con 0x7f44300d01b0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.324+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44280099b0 con 0x7f44300d01b0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.324+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442801c680 con 0x7f44300d01b0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.324+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44300afec0 con 0x7f44300d01b0 2023-11-28T11:17:55.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.324+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44300afbe0 con 0x7f44300d01b0 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.325+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44300b2600 con 0x7f44300d01b0 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.326+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f442802c070 con 0x7f44300d01b0 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.327+0000 7f4417fff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 0x7f441c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.327+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4428087850 con 0x7f44300d01b0 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.327+0000 7f442f7fe700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 0x7f441c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.328+0000 7f442f7fe700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 0x7f441c06a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b40 tx=0x7f4420009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.331+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4428053fc0 con 0x7f44300d01b0 2023-11-28T11:17:55.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.533+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4430068690 con 0x7f44300d01b0 2023-11-28T11:17:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.535+0000 7f4417fff700 1 -- 172.21.15.53:0/1536516236 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f44280575e0 con 0x7f44300d01b0 2023-11-28T11:17:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:55.536 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 msgr2=0x7f441c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 0x7f441c06a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b40 tx=0x7f4420009210 comp rx=0 tx=0).stop 2023-11-28T11:17:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 msgr2=0x7f443015c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f442800b370 tx=0x7f4428004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 shutdown_connections 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f441c0682f0 0x7f441c06a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300b16d0 0x7f4430157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44300d01b0 0x7f443015c2d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 --2- 172.21.15.53:0/1536516236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300b20b0 0x7f44301f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 >> 172.21.15.53:0/1536516236 conn(0x7f443009c3f0 msgr2=0x7f44301e6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 shutdown_connections 2023-11-28T11:17:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.537+0000 7f4435dff700 1 -- 172.21.15.53:0/1536516236 wait complete. 2023-11-28T11:17:55.556 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:17:55.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:17:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.756+0000 7f4786fff700 1 Processor -- start 2023-11-28T11:17:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.757+0000 7f4786fff700 1 -- start start 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f4786fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f4786fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47800d2320 0x7f47800cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f4786fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47800d1950 con 0x7f478015c9b0 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f4786fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800b5800 con 0x7f47800d2320 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f4786fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800b5300 con 0x7f4780161340 2023-11-28T11:17:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f477ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f478559c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47800d2320 0x7f47800cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f477ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60186/0 (socket says 172.21.15.53:60186) 2023-11-28T11:17:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.758+0000 7f477ffff700 1 -- 172.21.15.53:0/3099873799 learned_addr learned my addr 172.21.15.53:0/3099873799 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477ffff700 1 -- 172.21.15.53:0/3099873799 >> v2:172.21.15.157:3300/0 conn(0x7f47800d2320 msgr2=0x7f47800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477ffff700 1 --2- 172.21.15.53:0/3099873799 >> v2:172.21.15.157:3300/0 conn(0x7f47800d2320 0x7f47800cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477f7fe700 1 -- 172.21.15.53:0/3099873799 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732840295 0 0) 0x7f47800d1950 con 0x7f478015c9b0 2023-11-28T11:17:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477ffff700 1 -- 172.21.15.53:0/3099873799 >> v1:172.21.15.157:6789/0 conn(0x7f478015c9b0 legacy=0x7f47800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477ffff700 1 -- 172.21.15.53:0/3099873799 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800b5520 con 0x7f4780161340 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477ffff700 1 --2- 172.21.15.53:0/3099873799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f476c0098f0 tx=0x7f476c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4923cbc675c7857 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477f7fe700 1 -- 172.21.15.53:0/3099873799 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c011830 con 0x7f4780161340 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477f7fe700 1 -- 172.21.15.53:0/3099873799 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c011990 con 0x7f4780161340 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.759+0000 7f477f7fe700 1 -- 172.21.15.53:0/3099873799 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c011d20 con 0x7f4780161340 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 -- 172.21.15.53:0/3099873799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 msgr2=0x7f47800d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 --2- 172.21.15.53:0/3099873799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f476c0098f0 tx=0x7f476c009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 -- 172.21.15.53:0/3099873799 shutdown_connections 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 --2- 172.21.15.53:0/3099873799 >> v2:172.21.15.157:3300/0 conn(0x7f47800d2320 0x7f47800cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 --2- 172.21.15.53:0/3099873799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4780161340 0x7f47800d1de0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 -- 172.21.15.53:0/3099873799 >> 172.21.15.53:0/3099873799 conn(0x7f4780072600 msgr2=0x7f47800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 -- 172.21.15.53:0/3099873799 shutdown_connections 2023-11-28T11:17:55.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.760+0000 7f4786fff700 1 -- 172.21.15.53:0/3099873799 wait complete. 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.762+0000 7f4786fff700 1 Processor -- start 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.762+0000 7f4786fff700 1 -- start start 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f478015c9b0 0x7f47800c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4780161340 0x7f47801e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c7d00 con 0x7f47800d2320 2023-11-28T11:17:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c8620 con 0x7f4780161340 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4786fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4780143950 con 0x7f478015c9b0 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39890/0 (socket says 172.21.15.53:39890) 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 -- 172.21.15.53:0/2841672423 learned_addr learned my addr 172.21.15.53:0/2841672423 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f477ffff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f478015c9b0 0x7f47800c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f478559c700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4780161340 0x7f47801e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 -- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f478015c9b0 msgr2=0x7f47800c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f478015c9b0 0x7f47800c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 -- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4780161340 msgr2=0x7f47801e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4780161340 0x7f47801e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.763+0000 7f4784d9b700 1 -- 172.21.15.53:0/2841672423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009550 con 0x7f47800d2320 2023-11-28T11:17:55.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f4784d9b700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f477400c450 tx=0x7f477400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740054e0 con 0x7f47800d2320 2023-11-28T11:17:55.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400ce60 con 0x7f47800d2320 2023-11-28T11:17:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740038c0 con 0x7f47800d2320 2023-11-28T11:17:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800d15e0 con 0x7f47800d2320 2023-11-28T11:17:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.764+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47800d1300 con 0x7f47800d2320 2023-11-28T11:17:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.765+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478015cd80 con 0x7f47800d2320 2023-11-28T11:17:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.766+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4774019030 con 0x7f47800d2320 2023-11-28T11:17:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.768+0000 7f477d7fa700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 0x7f476406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.768+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4774084eb0 con 0x7f47800d2320 2023-11-28T11:17:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.768+0000 7f477ffff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 0x7f476406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.769+0000 7f477ffff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 0x7f476406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f476c000c00 tx=0x7f476c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.772+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4774051710 con 0x7f47800d2320 2023-11-28T11:17:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.979+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47800745c0 con 0x7f47800d2320 2023-11-28T11:17:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.980+0000 7f477d7fa700 1 -- 172.21.15.53:0/2841672423 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f4774017090 con 0x7f47800d2320 2023-11-28T11:17:55.981 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:55.981 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 msgr2=0x7f476406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 0x7f476406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f476c000c00 tx=0x7f476c008030 comp rx=0 tx=0).stop 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 msgr2=0x7f4780143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f477400c450 tx=0x7f477400c760 comp rx=0 tx=0).stop 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 shutdown_connections 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47640682f0 0x7f476406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4780161340 0x7f47801e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47800d2320 0x7f4780143410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 --2- 172.21.15.53:0/2841672423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f478015c9b0 0x7f47800c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.982+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 >> 172.21.15.53:0/2841672423 conn(0x7f4780072600 msgr2=0x7f47800b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.983+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 shutdown_connections 2023-11-28T11:17:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:55.983+0000 7f4786fff700 1 -- 172.21.15.53:0/2841672423 wait complete. 2023-11-28T11:17:56.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:17:56.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:17:56.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:17:56.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:17:56.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:17:56.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:17:56.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:17:56.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:17:56.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.373+0000 7f108dd9d700 1 Processor -- start 2023-11-28T11:17:56.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.373+0000 7f108dd9d700 1 -- start start 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.375+0000 7f108dd9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f108815c6e0 0x7f10880cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.376+0000 7f108dd9d700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880adb70 con 0x7f10880ce500 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.376+0000 7f108dd9d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880c9210 con 0x7f10880b0c20 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.376+0000 7f108dd9d700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108815cab0 con 0x7f108815c6e0 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.376+0000 7f1087fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10880ce500 0x7f10880b0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46856/0 (socket says 172.21.15.53:46856) 2023-11-28T11:17:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.376+0000 7f1087fff700 1 -- 172.21.15.53:0/3329052638 learned_addr learned my addr 172.21.15.53:0/3329052638 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f108cd9b700 1 --2- 172.21.15.53:0/3329052638 >> v2:172.21.15.53:3300/0 conn(0x7f108815c6e0 0x7f10880cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f10877fe700 1 -- 172.21.15.53:0/3329052638 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269728626 0 0) 0x7f10880c9210 con 0x7f10880b0c20 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f10877fe700 1 -- 172.21.15.53:0/3329052638 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10700035b0 con 0x7f10880b0c20 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f108cd9b700 1 -- 172.21.15.53:0/3329052638 >> v1:172.21.15.157:6789/0 conn(0x7f10880b0c20 legacy=0x7f10880c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f108cd9b700 1 -- 172.21.15.53:0/3329052638 >> v1:172.21.15.53:6789/0 conn(0x7f10880ce500 legacy=0x7f10880b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.377+0000 7f108cd9b700 1 -- 172.21.15.53:0/3329052638 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880ad2d0 con 0x7f108815c6e0 2023-11-28T11:17:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.379+0000 7f108cd9b700 1 --2- 172.21.15.53:0/3329052638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f108815c6e0 0x7f10880cdfc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f107c009840 tx=0x7f107c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05a7b47a740f16f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.379+0000 7f47f1b9d700 1 Processor -- start 2023-11-28T11:17:56.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.380+0000 7f47f1b9d700 1 -- start start 2023-11-28T11:17:56.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.382+0000 7fe8fbfff700 1 Processor -- start 2023-11-28T11:17:56.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.382+0000 7f10877fe700 1 -- 172.21.15.53:0/3329052638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c004230 con 0x7f108815c6e0 2023-11-28T11:17:56.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.382+0000 7f10877fe700 1 -- 172.21.15.53:0/3329052638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c004390 con 0x7f108815c6e0 2023-11-28T11:17:56.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.382+0000 7f10877fe700 1 -- 172.21.15.53:0/3329052638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0036c0 con 0x7f108815c6e0 2023-11-28T11:17:56.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.382+0000 7fe8fbfff700 1 -- start start 2023-11-28T11:17:56.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f47f1b9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47ec0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f108dd9d700 1 -- 172.21.15.53:0/3329052638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f108815c6e0 msgr2=0x7f10880cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f108dd9d700 1 --2- 172.21.15.53:0/3329052638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f108815c6e0 0x7f10880cdfc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f107c009840 tx=0x7f107c009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f47f1b9d700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0cd2d0 con 0x7f47ec0ce080 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f47f1b9d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0cc700 con 0x7f47ec160c00 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.383+0000 7f47f1b9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0cc9a0 con 0x7f47ec0ae0d0 2023-11-28T11:17:56.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f47f0b9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47ec160c00 0x7f47ec0ad9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43572/0 (socket says 172.21.15.53:43572) 2023-11-28T11:17:56.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f47f139c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f47ec0ce080 0x7f47ec0ca220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46868/0 (socket says 172.21.15.53:46868) 2023-11-28T11:17:56.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f47f0b9b700 1 -- 172.21.15.53:0/3051182987 learned_addr learned my addr 172.21.15.53:0/3051182987 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f108dd9d700 1 -- 172.21.15.53:0/3329052638 shutdown_connections 2023-11-28T11:17:56.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f108dd9d700 1 --2- 172.21.15.53:0/3329052638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f108815c6e0 0x7f10880cdfc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090987001 0 0) 0x7f47ec0cd2d0 con 0x7f47ec0ce080 2023-11-28T11:17:56.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d40035b0 con 0x7f47ec0ce080 2023-11-28T11:17:56.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7f108dd9d700 1 -- 172.21.15.53:0/3329052638 >> 172.21.15.53:0/3329052638 conn(0x7f10880cb510 msgr2=0x7f10880cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7fe8fbfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7fe8fbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8f40b20b0 0x7fe8f40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.384+0000 7fe8fbfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8f40b29b0 0x7fe8f40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f108dd9d700 1 -- 172.21.15.53:0/3329052638 shutdown_connections 2023-11-28T11:17:56.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47ec0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732043373 0 0) 0x7f47d40035b0 con 0x7f47ec0ce080 2023-11-28T11:17:56.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 msgr2=0x7f47ec0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47ec0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f1a2ddff700 1 Processor -- start 2023-11-28T11:17:56.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 >> v1:172.21.15.157:6789/0 conn(0x7f47ec160c00 legacy=0x7f47ec0ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec0ccc80 con 0x7f47ec0ce080 2023-11-28T11:17:56.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f41ff1ff700 1 Processor -- start 2023-11-28T11:17:56.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.385+0000 7f108dd9d700 1 -- 172.21.15.53:0/3329052638 wait complete. 2023-11-28T11:17:56.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f1a2ddff700 1 -- start start 2023-11-28T11:17:56.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f47e40019f0 con 0x7f47ec0ce080 2023-11-28T11:17:56.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 >> v1:172.21.15.53:6789/0 conn(0x7f47ec0ce080 legacy=0x7f47ec0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47d4003a30 0x7f47d4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f41ff1ff700 1 -- start start 2023-11-28T11:17:56.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4004850 0x7f47d4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0cc9a0 con 0x7f47d4003a30 2023-11-28T11:17:56.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4005560 con 0x7f47d4004850 2023-11-28T11:17:56.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40056d0 con 0x7f47ec0ae0d0 2023-11-28T11:17:56.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.386+0000 7fe18f59d700 1 Processor -- start 2023-11-28T11:17:56.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.387+0000 7fe18f59d700 1 -- start start 2023-11-28T11:17:56.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.387+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4004850 0x7f47d4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.387+0000 7f47f139c700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f47ebfff700 1 -- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 msgr2=0x7f47d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fbfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f415cec0 con 0x7fe8f40b20b0 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fbfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f40b0230 con 0x7fe8f40b29b0 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fbfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40b4570 con 0x7fe8f40d01b0 2023-11-28T11:17:56.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fa59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8f40b29b0 0x7fe8f40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fa59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8f40b29b0 0x7fe8f40b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:37928/0 (socket says 172.21.15.53:37928) 2023-11-28T11:17:56.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8fa59c700 1 -- 172.21.15.53:0/3796216848 learned_addr learned my addr 172.21.15.53:0/3796216848 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7fe8f9d9b700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f1a2ddff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a280c1fd0 0x7f1a28161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f47ebfff700 1 -- 172.21.15.53:0/3051182987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47d4003a30 msgr2=0x7f47d4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f1a2ddff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a28164e60 0x7f1a281652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f1a2ddff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a28162210 con 0x7f1a280ce480 2023-11-28T11:17:56.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f1a2ddff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a28161760 con 0x7f1a280c1fd0 2023-11-28T11:17:56.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.388+0000 7f1a2ddff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a281649d0 con 0x7f1a28164e60 2023-11-28T11:17:56.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47d4003a30 0x7f47d4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 -- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 msgr2=0x7fe8f40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f40d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 -- 172.21.15.53:0/3796216848 >> v2:172.21.15.53:3300/0 conn(0x7fe8f40b20b0 msgr2=0x7fe8f40d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/3796216848 >> v2:172.21.15.53:3300/0 conn(0x7fe8f40b20b0 0x7fe8f40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 -- 172.21.15.53:0/3796216848 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f40b4850 con 0x7fe8f40b29b0 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8f9d9b700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f40d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f1a277fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1a280ce480 0x7f1a2806b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43586/0 (socket says 172.21.15.53:43586) 2023-11-28T11:17:56.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f1a277fe700 1 -- 172.21.15.53:0/119920049 learned_addr learned my addr 172.21.15.53:0/119920049 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f41ff1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41f806d4d0 0x7f41f80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f108dd9d700 1 Processor -- start 2023-11-28T11:17:56.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f47ebfff700 1 -- 172.21.15.53:0/3051182987 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40035b0 con 0x7f47d4004850 2023-11-28T11:17:56.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b29b0 0x7fe8f40b2e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4066e80 tx=0x7fe8e8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8821f48839f49e7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f1a27fff700 1 --2- 172.21.15.53:0/119920049 >> v2:172.21.15.157:3300/0 conn(0x7f1a28164e60 0x7f1a281652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f47f139c700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47ec0ae0d0 0x7f47d400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:56.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.389+0000 7f41ff1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7fe8f8d99700 1 -- 172.21.15.53:0/3796216848 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8011830 con 0x7fe8f40b29b0 2023-11-28T11:17:56.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7fe8f8d99700 1 -- 172.21.15.53:0/3796216848 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8011990 con 0x7fe8f40b29b0 2023-11-28T11:17:56.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7fe8f8d99700 1 -- 172.21.15.53:0/3796216848 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8011d20 con 0x7fe8f40b29b0 2023-11-28T11:17:56.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41effff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41f806d4d0 0x7f41f80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41fcf9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41f80b0670 0x7f41f80cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46872/0 (socket says 172.21.15.53:46872) 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41fcf9b700 1 -- 172.21.15.53:0/2215486730 learned_addr learned my addr 172.21.15.53:0/2215486730 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f1a27fff700 1 -- 172.21.15.53:0/119920049 >> v2:172.21.15.53:3300/0 conn(0x7f1a280c1fd0 msgr2=0x7f1a28161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f1a27fff700 1 --2- 172.21.15.53:0/119920049 >> v2:172.21.15.53:3300/0 conn(0x7f1a280c1fd0 0x7f1a28161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f1a27fff700 1 -- 172.21.15.53:0/119920049 >> v1:172.21.15.157:6789/0 conn(0x7f1a280ce480 legacy=0x7f1a2806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f1a267fc700 1 -- 172.21.15.53:0/119920049 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627446931 0 0) 0x7f1a28162210 con 0x7f1a280ce480 2023-11-28T11:17:56.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f80b0f80 con 0x7f41f80b0670 2023-11-28T11:17:56.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41fd79c700 1 --2- 172.21.15.53:0/2215486730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f1a27fff700 1 -- 172.21.15.53:0/119920049 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280c2450 con 0x7f1a28164e60 2023-11-28T11:17:56.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f108dd9d700 1 -- start start 2023-11-28T11:17:56.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f80ad6e0 con 0x7f41f806d4d0 2023-11-28T11:17:56.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.390+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80aca60 con 0x7f41f806b820 2023-11-28T11:17:56.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4004850 0x7f47d4004cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47e000a9e0 tx=0x7f47d4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb4f961e5f0c2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f41fd79c700 1 -- 172.21.15.53:0/2215486730 >> v2:172.21.15.157:3300/0 conn(0x7f41f806d4d0 msgr2=0x7f41f80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f41fd79c700 1 --2- 172.21.15.53:0/2215486730 >> v2:172.21.15.157:3300/0 conn(0x7f41f806d4d0 0x7f41f80b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f41fd79c700 1 -- 172.21.15.53:0/2215486730 >> v1:172.21.15.53:6789/0 conn(0x7f41f80b0670 legacy=0x7f41f80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f41fd79c700 1 -- 172.21.15.53:0/2215486730 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80acd40 con 0x7f41f806b820 2023-11-28T11:17:56.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f1a27fff700 1 --2- 172.21.15.53:0/119920049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a28164e60 0x7f1a281652a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806fa80 tx=0x7f1a2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6779069ad79691f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000a480 con 0x7f47d4004850 2023-11-28T11:17:56.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f41fd79c700 1 --2- 172.21.15.53:0/2215486730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f806bca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f906906ad7d05181 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7fe8fbfff700 1 -- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b29b0 msgr2=0x7fe8f40b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b29b0 0x7fe8f40b2e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4066e80 tx=0x7fe8e8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0004250 con 0x7f47d4004850 2023-11-28T11:17:56.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f47eb7fe700 1 -- 172.21.15.53:0/3051182987 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e00036a0 con 0x7f47d4004850 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.391+0000 7f108dd9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10880b0c20 0x7f10881fe520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108dd9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10880ce500 0x7f10880c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108dd9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 0x7f10880c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108dd9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880c6bb0 con 0x7f10880ce500 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108dd9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881fea60 con 0x7f10880b0c20 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108dd9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880c2ef0 con 0x7f108815b870 2023-11-28T11:17:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18f59d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 0x7fe18003aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18f59d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18f59d700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800e2cb0 con 0x7fe1800e28e0 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18f59d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18003b660 con 0x7fe18003b1e0 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18f59d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800e63e0 con 0x7fe18004cd70 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f41ef7fe700 1 -- 172.21.15.53:0/2215486730 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000e070 con 0x7f41f806b820 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f41ef7fe700 1 -- 172.21.15.53:0/2215486730 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f000bbd0 con 0x7f41f806b820 2023-11-28T11:17:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 -- 172.21.15.53:0/3796216848 shutdown_connections 2023-11-28T11:17:56.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f40d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/3796216848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b29b0 0x7fe8f40b2e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/3796216848 >> v2:172.21.15.53:3300/0 conn(0x7fe8f40b20b0 0x7fe8f40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 -- 172.21.15.53:0/3796216848 >> 172.21.15.53:0/3796216848 conn(0x7fe8f409c3f0 msgr2=0x7fe8f40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f41ef7fe700 1 -- 172.21.15.53:0/2215486730 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000e070 con 0x7f41f806b820 2023-11-28T11:17:56.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f1a267fc700 1 -- 172.21.15.53:0/119920049 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000e070 con 0x7f1a28164e60 2023-11-28T11:17:56.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f1a267fc700 1 -- 172.21.15.53:0/119920049 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2000bbd0 con 0x7f1a28164e60 2023-11-28T11:17:56.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 -- 172.21.15.53:0/3796216848 shutdown_connections 2023-11-28T11:17:56.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f1a267fc700 1 -- 172.21.15.53:0/119920049 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20012430 con 0x7f1a28164e60 2023-11-28T11:17:56.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe8fbfff700 1 -- 172.21.15.53:0/3796216848 wait complete. 2023-11-28T11:17:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe1907ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7fe18dd9a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 0x7fe18003aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7fe1907ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:37966/0 (socket says 172.21.15.53:37966) 2023-11-28T11:17:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7fe1907ff700 1 -- 172.21.15.53:0/3490184963 learned_addr learned my addr 172.21.15.53:0/3490184963 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4004850 msgr2=0x7f47d4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f47f1b9d700 1 --2- 172.21.15.53:0/3051182987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4004850 0x7f47d4004cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47e000a9e0 tx=0x7f47d4005880 comp rx=0 tx=0).stop 2023-11-28T11:17:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7fea769ff700 1 Processor -- start 2023-11-28T11:17:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108d59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 0x7f10880c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108d59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 0x7f10880c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60246/0 (socket says 172.21.15.53:60246) 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108d59c700 1 -- 172.21.15.53:0/1731162769 learned_addr learned my addr 172.21.15.53:0/1731162769 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.392+0000 7f108cd9b700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10880b0c20 0x7f10881fe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f108d59c700 1 -- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10880b0c20 msgr2=0x7f10881fe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f108d59c700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10880b0c20 0x7f10881fe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f1087fff700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10880ce500 0x7f10880c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f108d59c700 1 -- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10880ce500 msgr2=0x7f10880c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f47f139c700 1 -- 172.21.15.53:0/3051182987 reap_dead start 2023-11-28T11:17:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 msgr2=0x7f41f806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f41ff1ff700 1 --2- 172.21.15.53:0/2215486730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f806bca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.393+0000 7f108d59c700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10880ce500 0x7f10880c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f108d59c700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0094a0 con 0x7f108815b870 2023-11-28T11:17:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 -- 172.21.15.53:0/119920049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a28164e60 msgr2=0x7f1a281652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/119920049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a28164e60 0x7f1a281652a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806fa80 tx=0x7f1a2000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fe18dd9a700 1 -- 172.21.15.53:0/3490184963 >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 msgr2=0x7fe1800065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fe18dd9a700 1 --2- 172.21.15.53:0/3490184963 >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fea769ff700 1 -- start start 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 -- 172.21.15.53:0/119920049 shutdown_connections 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/119920049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a28164e60 0x7f1a281652a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/119920049 >> v2:172.21.15.53:3300/0 conn(0x7f1a280c1fd0 0x7f1a28161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1a2ddff700 1 -- 172.21.15.53:0/119920049 >> 172.21.15.53:0/119920049 conn(0x7f1a280cb670 msgr2=0x7f1a280cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f1087fff700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10880ce500 0x7f10880c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fe18dd9a700 1 -- 172.21.15.53:0/3490184963 >> v1:172.21.15.53:6789/0 conn(0x7fe1800e28e0 legacy=0x7fe180055280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fe18dd9a700 1 -- 172.21.15.53:0/3490184963 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800e6630 con 0x7fe18004cd70 2023-11-28T11:17:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7fe1907ff700 1 --2- 172.21.15.53:0/3490184963 >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3051182987 shutdown_connections 2023-11-28T11:17:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3051182987 >> 172.21.15.53:0/3051182987 conn(0x7f47ec0cb510 msgr2=0x7f47ec0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 shutdown_connections 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f41ff1ff700 1 --2- 172.21.15.53:0/2215486730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f806bca0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f41ff1ff700 1 --2- 172.21.15.53:0/2215486730 >> v2:172.21.15.157:3300/0 conn(0x7f41f806d4d0 0x7f41f80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 >> 172.21.15.53:0/2215486730 conn(0x7f41f80cb670 msgr2=0x7f41f80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.394+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 shutdown_connections 2023-11-28T11:17:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7f1a2ddff700 1 -- 172.21.15.53:0/119920049 shutdown_connections 2023-11-28T11:17:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7f1a2ddff700 1 -- 172.21.15.53:0/119920049 wait complete. 2023-11-28T11:17:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7fe8fbfff700 1 Processor -- start 2023-11-28T11:17:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.396+0000 7f108d59c700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 0x7f10880c4030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f108000c760 tx=0x7f108000cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.396+0000 7fe8fbfff700 1 -- start start 2023-11-28T11:17:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.396+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080003e30 con 0x7f108815b870 2023-11-28T11:17:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.396+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10880ad890 con 0x7f108815b870 2023-11-28T11:17:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7f1a2ddff700 1 Processor -- start 2023-11-28T11:17:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.396+0000 7fe18dd9a700 1 --2- 172.21.15.53:0/3490184963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 0x7fe18003aca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe178009840 tx=0x7fe178009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f2a33143a6ece7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3051182987 shutdown_connections 2023-11-28T11:17:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3051182987 wait complete. 2023-11-28T11:17:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.395+0000 7f41ff1ff700 1 -- 172.21.15.53:0/2215486730 wait complete. 2023-11-28T11:17:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10880ad5b0 con 0x7f108815b870 2023-11-28T11:17:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080005070 con 0x7f108815b870 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080005420 con 0x7f108815b870 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7fe18d599700 1 -- 172.21.15.53:0/3490184963 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178004230 con 0x7fe18004cd70 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7fe18d599700 1 -- 172.21.15.53:0/3490184963 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178004390 con 0x7fe18004cd70 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7fe18d599700 1 -- 172.21.15.53:0/3490184963 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780036a0 con 0x7fe18004cd70 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7fea6ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fea700ce480 0x7fea7006b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43590/0 (socket says 172.21.15.53:43590) 2023-11-28T11:17:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7fea6ffff700 1 -- 172.21.15.53:0/4047933334 learned_addr learned my addr 172.21.15.53:0/4047933334 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f47f1b9d700 1 Processor -- start 2023-11-28T11:17:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7fea769ff700 1 --2- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea70161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f47f1b9d700 1 -- start start 2023-11-28T11:17:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7fe8fbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 0x7fe8f40b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.397+0000 7f1a2ddff700 1 -- start start 2023-11-28T11:17:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 0x7f1a281fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 0x7f1a2814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a28164e60 0x7f1a280c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281fab70 con 0x7f1a28164e60 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280b1a20 con 0x7f1a280ce480 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.398+0000 7f1a2ddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280b3720 con 0x7f1a280c1fd0 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f40b29b0 0x7fe8f42026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8f9d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 0x7fe8f40b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8f9d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 0x7fe8f40b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:37976/0 (socket says 172.21.15.53:37976) 2023-11-28T11:17:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7f41ff1ff700 1 Processor -- start 2023-11-28T11:17:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fbfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f4201e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe18f59d700 1 -- 172.21.15.53:0/3490184963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 msgr2=0x7fe18003aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe18f59d700 1 --2- 172.21.15.53:0/3490184963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 0x7fe18003aca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe178009840 tx=0x7fe178009b50 comp rx=0 tx=0).stop 2023-11-28T11:17:56.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7f1a26ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 0x7f1a2814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7f1a277fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 0x7f1a281fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8f9d9b700 1 -- 172.21.15.53:0/523116140 learned_addr learned my addr 172.21.15.53:0/523116140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4154260 con 0x7fe8f40b29b0 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4202c20 con 0x7fe8f40b20b0 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4202e80 con 0x7fe8f40d01b0 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f4201e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7f1a277fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 0x7f1a281fa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60260/0 (socket says 172.21.15.53:60260) 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f1a277fe700 1 -- 172.21.15.53:0/3139280234 learned_addr learned my addr 172.21.15.53:0/3139280234 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea70162210 con 0x7fea70164e60 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea70161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 0x7f47ec1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ec162ca0 0x7f47ec0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ec0b1e00 0x7f47ec161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0b4670 con 0x7f47ec0b1e00 2023-11-28T11:17:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec1f77b0 con 0x7f47ec162ca0 2023-11-28T11:17:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fe18f59d700 1 -- 172.21.15.53:0/3490184963 shutdown_connections 2023-11-28T11:17:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fe18f59d700 1 --2- 172.21.15.53:0/3490184963 >> v2:172.21.15.157:3300/0 conn(0x7fe18003b1e0 0x7fe1800065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f1b9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec161390 con 0x7f47d4004850 2023-11-28T11:17:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f1a26ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 0x7f1a2814c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:37986/0 (socket says 172.21.15.53:37986) 2023-11-28T11:17:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fe18f59d700 1 --2- 172.21.15.53:0/3490184963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18004cd70 0x7fe18003aca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fe18f59d700 1 -- 172.21.15.53:0/3490184963 >> 172.21.15.53:0/3490184963 conn(0x7fe180055ca0 msgr2=0x7fe1800560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.399+0000 7f41ff1ff700 1 -- start start 2023-11-28T11:17:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f0b9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 0x7f47ec1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f0b9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 0x7f47ec1f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60276/0 (socket says 172.21.15.53:60276) 2023-11-28T11:17:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47f0b9b700 1 -- 172.21.15.53:0/3557264445 learned_addr learned my addr 172.21.15.53:0/3557264445 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ec162ca0 0x7f47ec0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea701649d0 con 0x7fea700ce480 2023-11-28T11:17:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7f1a27fff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a28164e60 0x7f1a280c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.400+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70161760 con 0x7fea700c1fd0 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 -- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 msgr2=0x7fe8f40b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 0x7fe8f40b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 -- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f40b29b0 msgr2=0x7fe8f42026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f40b29b0 0x7fe8f42026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e4009a70 con 0x7fe8f40d01b0 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8fa59c700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f4201e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8016000 tx=0x7fe8e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8f9d9b700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8f40b20b0 0x7fe8f40b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004510 con 0x7fe8f40d01b0 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e80036a0 con 0x7fe8f40d01b0 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8003a10 con 0x7fe8f40d01b0 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e8009550 con 0x7fe8f40d01b0 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f1a27fff700 1 -- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 msgr2=0x7f1a281fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f1a27fff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 0x7f1a281fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f1a27fff700 1 -- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 msgr2=0x7f1a2814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f1a27fff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 0x7f1a2814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f1a27fff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20009a70 con 0x7f1a28164e60 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 -- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ec162ca0 msgr2=0x7f47ec0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47ec162ca0 0x7f47ec0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 -- 172.21.15.53:0/3557264445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ec0b1e00 msgr2=0x7f47ec161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ec0b1e00 0x7f47ec161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec160fd0 con 0x7f47d4004850 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f47f0b9b700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 0x7f47ec1f6830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c580 tx=0x7f47dc00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc005d70 con 0x7f47d4004850 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc0070d0 con 0x7f47d4004850 2023-11-28T11:17:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc007480 con 0x7f47d4004850 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec0ccf60 con 0x7f47d4004850 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fea6f7fe700 1 -- 172.21.15.53:0/4047933334 >> v1:172.21.15.157:6789/0 conn(0x7fea700ce480 legacy=0x7fea7006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fea6f7fe700 1 -- 172.21.15.53:0/4047933334 >> v1:172.21.15.53:6789/0 conn(0x7fea70164e60 legacy=0x7fea700b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7fea6f7fe700 1 -- 172.21.15.53:0/4047933334 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7006c020 con 0x7fea700c1fd0 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f41ff1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f8151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.401+0000 7f41ff1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41f806d4d0 0x7f41f8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41fcf9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f8151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41fcf9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f8151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60278/0 (socket says 172.21.15.53:60278) 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41ff1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f80b0670 0x7f41f8070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41ff1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f81519f0 con 0x7f41f806d4d0 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41ff1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80b1a20 con 0x7f41f80b0670 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41ff1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f81520d0 con 0x7f41f806b820 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f1a26ffd700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a280ce480 0x7f1a2814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f1a277fe700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a280c1fd0 0x7f1a281fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41fcf9b700 1 -- 172.21.15.53:0/372920459 learned_addr learned my addr 172.21.15.53:0/372920459 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f41fd79c700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f80b0670 0x7f41f8070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.402+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1080010070 con 0x7f108815b870 2023-11-28T11:17:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1068000fc0 con 0x7f108815b870 2023-11-28T11:17:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7f10857fa700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10740682f0 0x7f107406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1080085760 con 0x7f108815b870 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f10857fa700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f1074070aa0 0x7f1074072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1074073560 con 0x7f1074070aa0 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1080051fc0 con 0x7f108815b870 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f108cd9b700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10740682f0 0x7f107406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f1087fff700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f1074070aa0 0x7f1074072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f40afec0 con 0x7fe8f40d01b0 2023-11-28T11:17:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7f1a27fff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a28164e60 0x7f1a280c6c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a20016000 tx=0x7f1a20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7fe18f59d700 1 -- 172.21.15.53:0/3490184963 shutdown_connections 2023-11-28T11:17:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f47f1b9d700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec06b530 con 0x7f47d4004850 2023-11-28T11:17:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47ec161160 con 0x7f47d4004850 2023-11-28T11:17:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea70161220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f119eb3974e03491 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f41fd79c700 1 -- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 msgr2=0x7f41f8151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f41fd79c700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41f806b820 0x7f41f8151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f41fd79c700 1 -- 172.21.15.53:0/372920459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41f806d4d0 msgr2=0x7f41f8070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f41fd79c700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41f806d4d0 0x7f41f8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.403+0000 7f41fd79c700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009a70 con 0x7f41f80b0670 2023-11-28T11:17:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7fea6effd700 1 -- 172.21.15.53:0/4047933334 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64004230 con 0x7fea700c1fd0 2023-11-28T11:17:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7f108cd9b700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10740682f0 0x7f107406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10880675b0 tx=0x7f107c010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7fea6effd700 1 -- 172.21.15.53:0/4047933334 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64004390 con 0x7fea700c1fd0 2023-11-28T11:17:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7fea6effd700 1 -- 172.21.15.53:0/4047933334 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640036a0 con 0x7fea700c1fd0 2023-11-28T11:17:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.404+0000 7fe18f59d700 1 -- 172.21.15.53:0/3490184963 wait complete. 2023-11-28T11:17:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.405+0000 7f41fd79c700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f80b0670 0x7f41f8070e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f806fa30 tx=0x7f41f0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007000 con 0x7f41f80b0670 2023-11-28T11:17:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0020070 con 0x7f41f80b0670 2023-11-28T11:17:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00276c0 con 0x7f41f80b0670 2023-11-28T11:17:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.406+0000 7f41ff1ff700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f80ad300 con 0x7f41f80b0670 2023-11-28T11:17:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8f40afbe0 con 0x7fe8f40d01b0 2023-11-28T11:17:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20004800 con 0x7f1a28164e60 2023-11-28T11:17:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20020070 con 0x7f1a28164e60 2023-11-28T11:17:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a200276c0 con 0x7f1a28164e60 2023-11-28T11:17:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1087fff700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f1074070aa0 0x7f1074072eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a28161970 con 0x7f1a28164e60 2023-11-28T11:17:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f41ff1ff700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f80ad020 con 0x7f41f80b0670 2023-11-28T11:17:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2806c020 con 0x7f1a28164e60 2023-11-28T11:17:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47dc019030 con 0x7f47d4004850 2023-11-28T11:17:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 msgr2=0x7fea70161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.407+0000 7fea769ff700 1 --2- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea70161220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).stop 2023-11-28T11:17:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41f815caf0 con 0x7f41f80b0670 2023-11-28T11:17:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== osd.0 v2:172.21.15.53:6812/132689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1074073560 con 0x7f1074070aa0 2023-11-28T11:17:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f47e97fa700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47d80682f0 0x7f47d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fe18f59d700 1 Processor -- start 2023-11-28T11:17:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 shutdown_connections 2023-11-28T11:17:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47dc0850c0 con 0x7f47d4004850 2023-11-28T11:17:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fea769ff700 1 --2- 172.21.15.53:0/4047933334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea70161220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 >> 172.21.15.53:0/4047933334 conn(0x7fea700cb670 msgr2=0x7fea700cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fe18f59d700 1 -- start start 2023-11-28T11:17:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a08000fc0 con 0x7f1a28164e60 2023-11-28T11:17:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f47e97fa700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f47d8070ba0 0x7f47d8072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47d8073660 con 0x7f47d8070ba0 2023-11-28T11:17:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47dc0876e0 con 0x7f47d4004850 2023-11-28T11:17:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.409+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8e800f050 con 0x7fe8f40d01b0 2023-11-28T11:17:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.410+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 shutdown_connections 2023-11-28T11:17:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.410+0000 7fea769ff700 1 -- 172.21.15.53:0/4047933334 wait complete. 2023-11-28T11:17:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.410+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f41f0027820 con 0x7f41f80b0670 2023-11-28T11:17:56.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 0x7fe1800dd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18004cd70 0x7fe18003e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800e28e0 0x7fe1800d7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18003d140 con 0x7fe18004cd70 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18003d3b0 con 0x7fe1800e28e0 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18f59d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18003d690 con 0x7fe18003b1e0 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18e59b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 0x7fe1800dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18e59b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 0x7fe1800dd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60292/0 (socket says 172.21.15.53:60292) 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18e59b700 1 -- 172.21.15.53:0/2525905882 learned_addr learned my addr 172.21.15.53:0/2525905882 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe18dd9a700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18004cd70 0x7fe18003e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe1907ff700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800e28e0 0x7fe1800d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 -- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800e28e0 msgr2=0x7fe1800d7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800e28e0 0x7fe1800d7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 -- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18004cd70 msgr2=0x7fe18003e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18004cd70 0x7fe18003e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18804ec80 con 0x7fe18003b1e0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe1907ff700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800e28e0 0x7fe1800d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe18e59b700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 0x7fe1800dd570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1840043a0 tx=0x7fe184014330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.410+0000 7f47f139c700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f47d8070ba0 0x7f47d8072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.410+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47d80682f0 0x7f47d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7f47f139c700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f47d8070ba0 0x7f47d8072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7f47ebfff700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47d80682f0 0x7f47d806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47ec1637b0 tx=0x7f47e0012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== osd.4 v2:172.21.15.157:6800/132917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f47d8073660 con 0x7f47d8070ba0 2023-11-28T11:17:56.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7f41ed7fa700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41e40682f0 0x7f41e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe8f27fc700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe8e0068340 0x7fe8e006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.411+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8e8086d60 con 0x7fe8f40d01b0 2023-11-28T11:17:56.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe8f27fc700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7fe8e0070b70 0x7fe8e0072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8e0073630 con 0x7fe8e0070b70 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.412+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41f0022070 con 0x7f41f80b0670 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184017070 con 0x7fe18003b1e0 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184014b50 con 0x7fe18003b1e0 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7f41ed7fa700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f41e4070b20 0x7f41e4072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41e40735e0 con 0x7f41e4070b20 2023-11-28T11:17:56.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f008d440 con 0x7f41f80b0670 2023-11-28T11:17:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184010850 con 0x7fe18003b1e0 2023-11-28T11:17:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7f41fcf9b700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41e40682f0 0x7f41e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7f41effff700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f41e4070b20 0x7f41e4072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.413+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1780094a0 con 0x7fe18003b1e0 2023-11-28T11:17:56.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.414+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1800e6bf0 con 0x7fe18003b1e0 2023-11-28T11:17:56.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.414+0000 7fea769ff700 1 Processor -- start 2023-11-28T11:17:56.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.414+0000 7f41effff700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f41e4070b20 0x7f41e4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.414+0000 7f41fcf9b700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41e40682f0 0x7f41e406a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41f81602c0 tx=0x7f41e8001080 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.414+0000 7fea769ff700 1 -- start start 2023-11-28T11:17:56.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.415+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1800e6910 con 0x7fe18003b1e0 2023-11-28T11:17:56.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.415+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== osd.6 v2:172.21.15.157:6815/132919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f41e40735e0 con 0x7f41e4070b20 2023-11-28T11:17:56.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.416+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1840109b0 con 0x7fe18003b1e0 2023-11-28T11:17:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.416+0000 7fea769ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea7015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.416+0000 7fea769ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea700ce480 0x7fea700b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.416+0000 7fea769ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70164e60 0x7fea700c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea7015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea7015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60296/0 (socket says 172.21.15.53:60296) 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea6ffff700 1 -- 172.21.15.53:0/2086065395 learned_addr learned my addr 172.21.15.53:0/2086065395 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea74f9c700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70164e60 0x7fea700c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700b3e90 con 0x7fea700ce480 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c2960 con 0x7fea70164e60 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700b3280 con 0x7fea700c1fd0 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.415+0000 7fe8f959a700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7fe8e0070b70 0x7fe8e0072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.415+0000 7fe8f9d9b700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe8e0068340 0x7fe8e006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fe17effd700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe16c0683c0 0x7fe16c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe18408def0 con 0x7fe18003b1e0 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fe17effd700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7fe16c070c70 0x7fe16c073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16c073730 con 0x7fe16c070c70 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18408e2a0 con 0x7fe18003b1e0 2023-11-28T11:17:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea700ce480 0x7fea700b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.415+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a20027820 con 0x7f1a28164e60 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7f1a0ffff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a140683c0 0x7f1a1406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a20022070 con 0x7f1a28164e60 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7f1a277fe700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a140683c0 0x7f1a1406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.417+0000 7f1a0ffff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7f1a14070bf0 0x7f1a14073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a140736b0 con 0x7f1a14070bf0 2023-11-28T11:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8e8053740 con 0x7fe8f40d01b0 2023-11-28T11:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.418+0000 7fe8f9d9b700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe8e0068340 0x7fe8e006a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4071860 tx=0x7fe8e400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7f1a26ffd700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7f1a14070bf0 0x7f1a14073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a20004960 con 0x7f1a28164e60 2023-11-28T11:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fe18dd9a700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe16c0683c0 0x7fe16c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7f1a277fe700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a140683c0 0x7f1a1406a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c000f60 tx=0x7f1a1c007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fe1907ff700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7fe16c070c70 0x7fe16c073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea6ffff700 1 -- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70164e60 msgr2=0x7fea700c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea6ffff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70164e60 0x7fea700c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea6ffff700 1 -- 172.21.15.53:0/2086065395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea700ce480 msgr2=0x7fea700b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea6ffff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea700ce480 0x7fea700b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea6ffff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640094a0 con 0x7fea700c1fd0 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.419+0000 7fea74f9c700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70164e60 0x7fea700c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7fe8f959a700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7fe8e0070b70 0x7fe8e0072f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7f1a26ffd700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7f1a14070bf0 0x7f1a14073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7fe18dd9a700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe16c0683c0 0x7fe16c06a870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7fe1907ff700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7fe16c070c70 0x7fe16c073080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea700ce480 0x7fea700b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7fea6ffff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea7015c160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea6000c520 tx=0x7fea6000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60005d70 con 0x7fea700c1fd0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea600070d0 con 0x7fea700c1fd0 2023-11-28T11:17:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60007440 con 0x7fea700c1fd0 2023-11-28T11:17:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea70161970 con 0x7fea700c1fd0 2023-11-28T11:17:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.421+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea700c2420 con 0x7fea700c1fd0 2023-11-28T11:17:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7f87c739d700 1 Processor -- start 2023-11-28T11:17:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.420+0000 7f87c739d700 1 -- start start 2023-11-28T11:17:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.422+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== osd.3 v2:172.21.15.53:6823/132695 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe16c073730 con 0x7fe16c070c70 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.422+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea70161c50 con 0x7fea700c1fd0 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.423+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea60019030 con 0x7fea700c1fd0 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.424+0000 7f87c739d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87b80367f0 0x7f87b80e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.424+0000 7f87c739d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.424+0000 7f87c739d700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b80e8260 con 0x7f87b8036bc0 2023-11-28T11:17:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.424+0000 7f87c739d700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b804f9b0 con 0x7f87b80367f0 2023-11-28T11:17:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.424+0000 7f87c739d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8051170 con 0x7f87b80544f0 2023-11-28T11:17:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7fea6cff9700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea5c0682f0 0x7fea5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7f87c5b9a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7f87c5b9a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60306/0 (socket says 172.21.15.53:60306) 2023-11-28T11:17:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7f87c5b9a700 1 -- 172.21.15.53:0/3165879413 learned_addr learned my addr 172.21.15.53:0/3165879413 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea60084f30 con 0x7fea700c1fd0 2023-11-28T11:17:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7fea6cff9700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fea5c070b20 0x7fea5c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f87c639b700 1 --2- 172.21.15.53:0/3165879413 >> v2:172.21.15.53:3300/0 conn(0x7f87b80367f0 0x7f87b80e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f87c5b9a700 1 -- 172.21.15.53:0/3165879413 >> v2:172.21.15.53:3300/0 conn(0x7f87b80367f0 msgr2=0x7f87b80e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/3165879413 >> v2:172.21.15.53:3300/0 conn(0x7f87b80367f0 0x7f87b80e6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f87c5b9a700 1 -- 172.21.15.53:0/3165879413 >> v1:172.21.15.53:6789/0 conn(0x7f87b8036bc0 legacy=0x7f87b80e7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f87c5b9a700 1 -- 172.21.15.53:0/3165879413 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8051450 con 0x7f87b80544f0 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea5c0735e0 con 0x7fea5c070b20 2023-11-28T11:17:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== osd.2 v2:172.21.15.53:6807/132691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1a140736b0 con 0x7f1a14070bf0 2023-11-28T11:17:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.425+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea6008a540 con 0x7fea700c1fd0 2023-11-28T11:17:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== osd.1 v2:172.21.15.53:6800/132688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe8e0073630 con 0x7fe8e0070b70 2023-11-28T11:17:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/3165879413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87b00098f0 tx=0x7f87b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d7897a87a7c2ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7fea74f9c700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fea5c070b20 0x7fea5c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.426+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea5c0682f0 0x7fea5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7fea74f9c700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fea5c070b20 0x7fea5c072f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7fea6f7fe700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea5c0682f0 0x7fea5c06a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea64008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7f87c5399700 1 -- 172.21.15.53:0/3165879413 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004230 con 0x7f87b80544f0 2023-11-28T11:17:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7f87c5399700 1 -- 172.21.15.53:0/3165879413 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b0004390 con 0x7f87b80544f0 2023-11-28T11:17:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.427+0000 7f87c5399700 1 -- 172.21.15.53:0/3165879413 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00036a0 con 0x7f87b80544f0 2023-11-28T11:17:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.428+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== osd.5 v2:172.21.15.157:6808/132918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fea5c0735e0 con 0x7fea5c070b20 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.428+0000 7f87c739d700 1 -- 172.21.15.53:0/3165879413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 msgr2=0x7f87b804f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.428+0000 7f87c739d700 1 --2- 172.21.15.53:0/3165879413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87b00098f0 tx=0x7f87b0009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 -- 172.21.15.53:0/3165879413 shutdown_connections 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 --2- 172.21.15.53:0/3165879413 >> v2:172.21.15.53:3300/0 conn(0x7f87b80367f0 0x7f87b80e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 --2- 172.21.15.53:0/3165879413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80544f0 0x7f87b804f470 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 -- 172.21.15.53:0/3165879413 >> 172.21.15.53:0/3165879413 conn(0x7f87b80517f0 msgr2=0x7f87b8051bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 -- 172.21.15.53:0/3165879413 shutdown_connections 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.429+0000 7f87c739d700 1 -- 172.21.15.53:0/3165879413 wait complete. 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.432+0000 7f87c739d700 1 Processor -- start 2023-11-28T11:17:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.432+0000 7f87c739d700 1 -- start start 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.434+0000 7f87c739d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80367f0 0x7f87b804e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.434+0000 7f87c739d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b8036bc0 0x7f87b8037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c639b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80367f0 0x7f87b804e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c639b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80367f0 0x7f87b804e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60316/0 (socket says 172.21.15.53:60316) 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c639b700 1 -- 172.21.15.53:0/1671892167 learned_addr learned my addr 172.21.15.53:0/1671892167 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c739d700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b80544f0 0x7f87b80e3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80e3970 con 0x7f87b80544f0 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80e29e0 con 0x7f87b8036bc0 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.435+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b803a7b0 con 0x7f87b80367f0 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.436+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b8036bc0 0x7f87b8037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.436+0000 7f87c5b9a700 1 -- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80367f0 msgr2=0x7f87b804e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.436+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b80367f0 0x7f87b804e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.436+0000 7f87c5b9a700 1 -- 172.21.15.53:0/1671892167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b80544f0 msgr2=0x7f87b80e3430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.437+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b80544f0 0x7f87b80e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.437+0000 7f87c5b9a700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009550 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.437+0000 7f87c5b9a700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b8036bc0 0x7f87b8037c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87b0015000 tx=0x7f87b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.438+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000b860 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.438+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000b9c0 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.438+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000b860 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.438+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b8054940 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.438+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b80e7630 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.439+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87b8050d90 con 0x7f87b8036bc0 2023-11-28T11:17:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.442+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87b0017020 con 0x7f87b8036bc0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.443+0000 7f87b6ffd700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87a4068350 0x7f87a406a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.444+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87b0060560 con 0x7f87b8036bc0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.444+0000 7f87c639b700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87a4068350 0x7f87a406a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.445+0000 7f87c639b700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87a4068350 0x7f87a406a800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00ba70 tx=0x7f87bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.445+0000 7f87b6ffd700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f87a4071860 0x7f87a4073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.445+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87a40743c0 con 0x7f87a4071860 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.446+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b0054810 con 0x7f87b8036bc0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.446+0000 7f87c85ff700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f87a4071860 0x7f87a4073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.447+0000 7f87c85ff700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f87a4071860 0x7f87a4073cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.448+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== osd.7 v2:172.21.15.157:6824/132962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f87a40743c0 con 0x7f87a4071860 2023-11-28T11:17:56.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.449+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41f80745c0 con 0x7f41e4070b20 2023-11-28T11:17:56.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.448+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 --> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47ec1614d0 con 0x7f47d8070ba0 2023-11-28T11:17:56.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41ed7fa700 1 -- 172.21.15.53:0/372920459 <== osd.6 v2:172.21.15.157:6815/132919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f41f80745c0 con 0x7f41e4070b20 2023-11-28T11:17:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f41e4070b20 msgr2=0x7f41e4072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f41e4070b20 0x7f41e4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41e40682f0 msgr2=0x7f41e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41e40682f0 0x7f41e406a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41f81602c0 tx=0x7f41e8001080 comp rx=0 tx=0).stop 2023-11-28T11:17:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f80b0670 msgr2=0x7f41f8070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.450+0000 7f41deffd700 1 --2- 172.21.15.53:0/372920459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41f80b0670 0x7f41f8070e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f806fa30 tx=0x7f41f0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.451+0000 7f41fd79c700 1 -- 172.21.15.53:0/372920459 reap_dead start 2023-11-28T11:17:56.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.451+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 shutdown_connections 2023-11-28T11:17:56.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.451+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 >> 172.21.15.53:0/372920459 conn(0x7f41f80cb670 msgr2=0x7f41f815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.451+0000 7f47e97fa700 1 -- 172.21.15.53:0/3557264445 <== osd.4 v2:172.21.15.157:6800/132917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47e40018d0 con 0x7f47d8070ba0 2023-11-28T11:17:56.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.451+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 shutdown_connections 2023-11-28T11:17:56.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.452+0000 7f41deffd700 1 -- 172.21.15.53:0/372920459 wait complete. 2023-11-28T11:17:56.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.452+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 --> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1068002d40 con 0x7f1074070aa0 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f47d8070ba0 msgr2=0x7f47d8072fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f47d8070ba0 0x7f47d8072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47d80682f0 msgr2=0x7f47d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47d80682f0 0x7f47d806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47ec1637b0 tx=0x7f47e0012000 comp rx=0 tx=0).stop 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 msgr2=0x7f47ec1f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 --2- 172.21.15.53:0/3557264445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d4004850 0x7f47ec1f6830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c580 tx=0x7f47dc00c940 comp rx=0 tx=0).stop 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47f139c700 1 -- 172.21.15.53:0/3557264445 reap_dead start 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 shutdown_connections 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 >> 172.21.15.53:0/3557264445 conn(0x7f47ec0cb510 msgr2=0x7f47ec06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.453+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 shutdown_connections 2023-11-28T11:17:56.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.454+0000 7f47d2ffd700 1 -- 172.21.15.53:0/3557264445 wait complete. 2023-11-28T11:17:56.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.454+0000 7f10857fa700 1 -- 172.21.15.53:0/1731162769 <== osd.0 v2:172.21.15.53:6812/132689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1068002d40 con 0x7f1074070aa0 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.454+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f1074070aa0 msgr2=0x7f1074072eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.454+0000 7f108dd9d700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f1074070aa0 0x7f1074072eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10740682f0 msgr2=0x7f107406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10740682f0 0x7f107406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10880675b0 tx=0x7f107c010000 comp rx=0 tx=0).stop 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 msgr2=0x7f10880c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 --2- 172.21.15.53:0/1731162769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f108815b870 0x7f10880c4030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f108000c760 tx=0x7f108000cb20 comp rx=0 tx=0).stop 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108d59c700 1 -- 172.21.15.53:0/1731162769 reap_dead start 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 shutdown_connections 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.455+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 >> 172.21.15.53:0/1731162769 conn(0x7f10880cb510 msgr2=0x7f10880706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.456+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 shutdown_connections 2023-11-28T11:17:56.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.456+0000 7f108dd9d700 1 -- 172.21.15.53:0/1731162769 wait complete. 2023-11-28T11:17:56.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.459+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 --> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1800e6ed0 con 0x7fe16c070c70 2023-11-28T11:17:56.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.461+0000 7fe17effd700 1 -- 172.21.15.53:0/2525905882 <== osd.3 v2:172.21.15.53:6823/132695 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1800e6ed0 con 0x7fe16c070c70 2023-11-28T11:17:56.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.460+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea700b4170 con 0x7fea5c070b20 2023-11-28T11:17:56.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.462+0000 7fea6cff9700 1 -- 172.21.15.53:0/2086065395 <== osd.5 v2:172.21.15.157:6808/132918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea700b4170 con 0x7fea5c070b20 2023-11-28T11:17:56.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.463+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8f40b2600 con 0x7fe8e0070b70 2023-11-28T11:17:56.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fea5c070b20 msgr2=0x7fea5c072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fea5c070b20 0x7fea5c072f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea5c0682f0 msgr2=0x7fea5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea5c0682f0 0x7fea5c06a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea64008030 comp rx=0 tx=0).stop 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 msgr2=0x7fea7015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 --2- 172.21.15.53:0/2086065395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea700c1fd0 0x7fea7015c160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea6000c520 tx=0x7fea6000c830 comp rx=0 tx=0).stop 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea74f9c700 1 -- 172.21.15.53:0/2086065395 reap_dead start 2023-11-28T11:17:56.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 --> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a08002d40 con 0x7f1a14070bf0 2023-11-28T11:17:56.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7f1a0ffff700 1 -- 172.21.15.53:0/3139280234 <== osd.2 v2:172.21.15.53:6807/132691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a08002d40 con 0x7f1a14070bf0 2023-11-28T11:17:56.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.464+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 shutdown_connections 2023-11-28T11:17:56.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 >> 172.21.15.53:0/2086065395 conn(0x7fea700cb670 msgr2=0x7fea7015fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe8f27fc700 1 -- 172.21.15.53:0/523116140 <== osd.1 v2:172.21.15.53:6800/132688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8f40b2600 con 0x7fe8e0070b70 2023-11-28T11:17:56.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7fe16c070c70 msgr2=0x7fe16c073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7fe16c070c70 0x7fe16c073080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7f1a14070bf0 msgr2=0x7f1a14073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7f1a14070bf0 0x7f1a14073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 shutdown_connections 2023-11-28T11:17:56.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fea769ff700 1 -- 172.21.15.53:0/2086065395 wait complete. 2023-11-28T11:17:56.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe16c0683c0 msgr2=0x7fe16c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe16c0683c0 0x7fe16c06a870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178010000 comp rx=0 tx=0).stop 2023-11-28T11:17:56.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 msgr2=0x7fe1800dd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.466+0000 7fe18f59d700 1 --2- 172.21.15.53:0/2525905882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18003b1e0 0x7fe1800dd570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1840043a0 tx=0x7fe184014330 comp rx=0 tx=0).stop 2023-11-28T11:17:56.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7fe8e0070b70 msgr2=0x7fe8e0072f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7fe8e0070b70 0x7fe8e0072f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe1907ff700 1 -- 172.21.15.53:0/2525905882 reap_dead start 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe8e0068340 msgr2=0x7fe8e006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe8e0068340 0x7fe8e006a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4071860 tx=0x7fe8e400b040 comp rx=0 tx=0).stop 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 msgr2=0x7fe8f4201e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 --2- 172.21.15.53:0/523116140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f40d01b0 0x7fe8f4201e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8016000 tx=0x7fe8e8004750 comp rx=0 tx=0).stop 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fa59c700 1 -- 172.21.15.53:0/523116140 reap_dead start 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 shutdown_connections 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 >> 172.21.15.53:0/523116140 conn(0x7fe8f409c3f0 msgr2=0x7fe8f40b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.467+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 shutdown_connections 2023-11-28T11:17:56.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7fe8fbfff700 1 -- 172.21.15.53:0/523116140 wait complete. 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 shutdown_connections 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 >> 172.21.15.53:0/2525905882 conn(0x7fe180055ca0 msgr2=0x7fe1800e09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a140683c0 msgr2=0x7f1a1406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a140683c0 0x7f1a1406a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c000f60 tx=0x7f1a1c007ea0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a28164e60 msgr2=0x7f1a280c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7f1a2ddff700 1 --2- 172.21.15.53:0/3139280234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a28164e60 0x7f1a280c6c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a20016000 tx=0x7f1a20004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7f1a27fff700 1 -- 172.21.15.53:0/3139280234 reap_dead start 2023-11-28T11:17:56.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.468+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 shutdown_connections 2023-11-28T11:17:56.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.469+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 shutdown_connections 2023-11-28T11:17:56.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.469+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 >> 172.21.15.53:0/3139280234 conn(0x7f1a280cb670 msgr2=0x7f1a2806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.469+0000 7fe18f59d700 1 -- 172.21.15.53:0/2525905882 wait complete. 2023-11-28T11:17:56.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.469+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 shutdown_connections 2023-11-28T11:17:56.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.469+0000 7f1a2ddff700 1 -- 172.21.15.53:0/3139280234 wait complete. 2023-11-28T11:17:56.496 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:56.497 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:56.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.484+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87b80055b0 con 0x7f87a4071860 2023-11-28T11:17:56.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.485+0000 7f87b6ffd700 1 -- 172.21.15.53:0/1671892167 <== osd.7 v2:172.21.15.157:6824/132962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87b80055b0 con 0x7f87a4071860 2023-11-28T11:17:56.498 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:17:56.498 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f87a4071860 msgr2=0x7f87a4073cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f87a4071860 0x7f87a4073cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87a4068350 msgr2=0x7f87a406a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f87a4068350 0x7f87a406a800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00ba70 tx=0x7f87bc008040 comp rx=0 tx=0).stop 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b8036bc0 msgr2=0x7f87b8037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.486+0000 7f87c739d700 1 --2- 172.21.15.53:0/1671892167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b8036bc0 0x7f87b8037c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87b0015000 tx=0x7f87b0004750 comp rx=0 tx=0).stop 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.487+0000 7f87c85ff700 1 -- 172.21.15.53:0/1671892167 reap_dead start 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.488+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 shutdown_connections 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.488+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 >> 172.21.15.53:0/1671892167 conn(0x7f87b80517f0 msgr2=0x7f87b8054d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.488+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 shutdown_connections 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.488+0000 7f87c739d700 1 -- 172.21.15.53:0/1671892167 wait complete. 2023-11-28T11:17:56.501 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:56.502 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:56.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:17:56.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:17:56.504 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:56.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:17:56.507 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:56.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:17:56.514 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:56.515 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:56.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:17:56.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:17:56.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.860+0000 7ff13cbff700 1 Processor -- start 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.861+0000 7ff13cbff700 1 -- start start 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13cbff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff13806d4f0 0x7ff1380b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13cbff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1380b0bb0 0x7ff13806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13cbff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1380b1010 con 0x7ff138158890 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13cbff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13815c490 con 0x7ff1380b0bb0 2023-11-28T11:17:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13cbff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13815c730 con 0x7ff13806d4f0 2023-11-28T11:17:56.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.864+0000 7ff136d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1380b0bb0 0x7ff13806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.863+0000 7ff13659c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff138158890 0x7ff1380ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43594/0 (socket says 172.21.15.53:43594) 2023-11-28T11:17:56.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.864+0000 7ff13659c700 1 -- 172.21.15.53:0/279316941 learned_addr learned my addr 172.21.15.53:0/279316941 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff12ffff700 1 --2- 172.21.15.53:0/279316941 >> v2:172.21.15.157:3300/0 conn(0x7ff13806d4f0 0x7ff1380b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff135d9b700 1 -- 172.21.15.53:0/279316941 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954025769 0 0) 0x7ff1380b1010 con 0x7ff138158890 2023-11-28T11:17:56.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff135d9b700 1 -- 172.21.15.53:0/279316941 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1240035b0 con 0x7ff138158890 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff136d9d700 1 -- 172.21.15.53:0/279316941 >> v2:172.21.15.157:3300/0 conn(0x7ff13806d4f0 msgr2=0x7ff1380b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff136d9d700 1 --2- 172.21.15.53:0/279316941 >> v2:172.21.15.157:3300/0 conn(0x7ff13806d4f0 0x7ff1380b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff136d9d700 1 -- 172.21.15.53:0/279316941 >> v1:172.21.15.157:6789/0 conn(0x7ff138158890 legacy=0x7ff1380ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.865+0000 7ff136d9d700 1 -- 172.21.15.53:0/279316941 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13815ca10 con 0x7ff1380b0bb0 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.866+0000 7ff136d9d700 1 --2- 172.21.15.53:0/279316941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13806b820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1380669d0 tx=0x7ff13000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49557491ed9631e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.866+0000 7ff135d9b700 1 -- 172.21.15.53:0/279316941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13000e070 con 0x7ff1380b0bb0 2023-11-28T11:17:56.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.867+0000 7ff135d9b700 1 -- 172.21.15.53:0/279316941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13000f250 con 0x7ff1380b0bb0 2023-11-28T11:17:56.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7ff13cbff700 1 -- 172.21.15.53:0/279316941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 msgr2=0x7ff13806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7ff13cbff700 1 --2- 172.21.15.53:0/279316941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13806b820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1380669d0 tx=0x7ff13000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:56.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7ff13cbff700 1 -- 172.21.15.53:0/279316941 shutdown_connections 2023-11-28T11:17:56.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7ff13cbff700 1 --2- 172.21.15.53:0/279316941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13806b820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7ff13cbff700 1 --2- 172.21.15.53:0/279316941 >> v2:172.21.15.157:3300/0 conn(0x7ff13806d4f0 0x7ff1380b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7ff13cbff700 1 -- 172.21.15.53:0/279316941 >> 172.21.15.53:0/279316941 conn(0x7ff1380cb670 msgr2=0x7ff1380cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.866+0000 7fb1eaf9d700 1 Processor -- start 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.866+0000 7fb1eaf9d700 1 -- start start 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 0x7fb1e416cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb1e408a5a0 0x7fb1e406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 0x7fb1e40b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1e979a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb1e408a5a0 0x7fb1e406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40b0ed0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e4169800 con 0x7fb1e406c790 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.868+0000 7fb1eaf9d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e416d520 con 0x7fb1e40b0510 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1e9f9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 0x7fb1e416cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1ea79c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 0x7fb1e40b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1ea79c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 0x7fb1e40b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60332/0 (socket says 172.21.15.53:60332) 2023-11-28T11:17:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1ea79c700 1 -- 172.21.15.53:0/2610643954 learned_addr learned my addr 172.21.15.53:0/2610643954 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1e9f9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 0x7fb1e416cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38044/0 (socket says 172.21.15.53:38044) 2023-11-28T11:17:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7ff13cbff700 1 -- 172.21.15.53:0/279316941 shutdown_connections 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.869+0000 7fb1e979a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb1e408a5a0 0x7fb1e406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39992/0 (socket says 172.21.15.53:39992) 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.870+0000 7fb1e979a700 1 -- 172.21.15.53:0/2610643954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 msgr2=0x7fb1e40b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.870+0000 7fb1e979a700 1 --2- 172.21.15.53:0/2610643954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 0x7fb1e40b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.870+0000 7fb1e979a700 1 -- 172.21.15.53:0/2610643954 >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 msgr2=0x7fb1e416cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.870+0000 7fb1e979a700 1 --2- 172.21.15.53:0/2610643954 >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 0x7fb1e416cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.870+0000 7fb1e979a700 1 -- 172.21.15.53:0/2610643954 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e416d800 con 0x7fb1e408a5a0 2023-11-28T11:17:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.871+0000 7fb1e979a700 1 --2- 172.21.15.53:0/2610643954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e408a5a0 0x7fb1e406b530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40675b0 tx=0x7fb1e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4aa35609298ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.872+0000 7fb1e8f99700 1 -- 172.21.15.53:0/2610643954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0014e90 con 0x7fb1e408a5a0 2023-11-28T11:17:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.872+0000 7fb1e8f99700 1 -- 172.21.15.53:0/2610643954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0014e90 con 0x7fb1e408a5a0 2023-11-28T11:17:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.873+0000 7f274359d700 1 Processor -- start 2023-11-28T11:17:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.873+0000 7f274359d700 1 -- start start 2023-11-28T11:17:56.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.871+0000 7ff13cbff700 1 -- 172.21.15.53:0/279316941 wait complete. 2023-11-28T11:17:56.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.873+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/2610643954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e408a5a0 msgr2=0x7fb1e406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.873+0000 7ff13cbff700 1 Processor -- start 2023-11-28T11:17:56.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.874+0000 7fb1e8f99700 1 -- 172.21.15.53:0/2610643954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e001c440 con 0x7fb1e408a5a0 2023-11-28T11:17:56.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.874+0000 7ff13cbff700 1 -- start start 2023-11-28T11:17:56.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.874+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/2610643954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e408a5a0 0x7fb1e406b530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40675b0 tx=0x7fb1e0009b20 comp rx=0 tx=0).stop 2023-11-28T11:17:56.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/2610643954 shutdown_connections 2023-11-28T11:17:56.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/2610643954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40b0510 0x7fb1e40b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/2610643954 >> v2:172.21.15.157:3300/0 conn(0x7fb1e406c790 0x7fb1e416cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/2610643954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e408a5a0 0x7fb1e406b530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/2610643954 >> 172.21.15.53:0/2610643954 conn(0x7fb1e40cb510 msgr2=0x7fb1e40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7f274359d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f273c160c00 0x7f273c0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7f274359d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/2610643954 shutdown_connections 2023-11-28T11:17:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.875+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/2610643954 wait complete. 2023-11-28T11:17:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f274359d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f273c0ad810 0x7f273c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f274359d700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c164670 con 0x7f273c0ad810 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f274359d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c160fd0 con 0x7f273c160c00 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f274359d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c165460 con 0x7f273c06b530 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f2741d9a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f2741d9a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60344/0 (socket says 172.21.15.53:60344) 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.876+0000 7f2741d9a700 1 -- 172.21.15.53:0/3151143968 learned_addr learned my addr 172.21.15.53:0/3151143968 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2742d9c700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.53:3300/0 conn(0x7f273c0ad810 0x7f273c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f274259b700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.157:3300/0 conn(0x7f273c160c00 0x7f273c0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2741d9a700 1 -- 172.21.15.53:0/3151143968 >> v2:172.21.15.157:3300/0 conn(0x7f273c160c00 msgr2=0x7f273c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2741d9a700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.157:3300/0 conn(0x7f273c160c00 0x7f273c0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2741d9a700 1 -- 172.21.15.53:0/3151143968 >> v2:172.21.15.53:3300/0 conn(0x7f273c0ad810 msgr2=0x7f273c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2741d9a700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.53:3300/0 conn(0x7f273c0ad810 0x7f273c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.877+0000 7f2741d9a700 1 -- 172.21.15.53:0/3151143968 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0ae1a0 con 0x7f273c06b530 2023-11-28T11:17:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 0x7ff138159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 0x7ff1381f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381f2590 con 0x7ff1380b0bb0 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13815a560 con 0x7ff13806d4f0 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13cbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13815a840 con 0x7ff138158890 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff13659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 0x7ff138159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.878+0000 7ff136d9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 0x7ff1381f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff13659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 0x7ff138159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38070/0 (socket says 172.21.15.53:38070) 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7f2741d9a700 1 --2- 172.21.15.53:0/3151143968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f273c0cb470 tx=0x7f272c009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75f8db1872426ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff13659c700 1 -- 172.21.15.53:0/3527849348 learned_addr learned my addr 172.21.15.53:0/3527849348 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 -- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 msgr2=0x7ff138159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 0x7ff138159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 -- 172.21.15.53:0/3527849348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 msgr2=0x7ff13815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 -- 172.21.15.53:0/3527849348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff130009a70 con 0x7ff138158890 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.879+0000 7ff136d9d700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 0x7ff1381f1400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff13001a000 tx=0x7ff130004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7f2741599700 1 -- 172.21.15.53:0/3151143968 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011830 con 0x7f273c06b530 2023-11-28T11:17:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7fb1eaf9d700 1 Processor -- start 2023-11-28T11:17:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7f2741599700 1 -- 172.21.15.53:0/3151143968 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c011990 con 0x7f273c06b530 2023-11-28T11:17:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7f2741599700 1 -- 172.21.15.53:0/3151143968 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011d20 con 0x7f273c06b530 2023-11-28T11:17:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7fb1eaf9d700 1 -- start start 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.881+0000 7f274359d700 1 -- 172.21.15.53:0/3151143968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 msgr2=0x7f273c0b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.881+0000 7f274359d700 1 --2- 172.21.15.53:0/3151143968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f273c0cb470 tx=0x7f272c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130009410 con 0x7ff138158890 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13000f050 con 0x7ff138158890 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.880+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130010440 con 0x7ff138158890 2023-11-28T11:17:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1380c9370 con 0x7ff138158890 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7f274359d700 1 -- 172.21.15.53:0/3151143968 shutdown_connections 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7f274359d700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.53:3300/0 conn(0x7f273c0ad810 0x7f273c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7f274359d700 1 --2- 172.21.15.53:0/3151143968 >> v2:172.21.15.157:3300/0 conn(0x7f273c160c00 0x7f273c0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7f274359d700 1 --2- 172.21.15.53:0/3151143968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c06b530 0x7f273c0b0a50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7f274359d700 1 -- 172.21.15.53:0/3151143968 >> 172.21.15.53:0/3151143968 conn(0x7f273c0cb510 msgr2=0x7f273c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13815ccf0 con 0x7ff138158890 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.882+0000 7fb1eaf9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7fb1eaf9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7fb1e9f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7fb1e9f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38074/0 (socket says 172.21.15.53:38074) 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7fb1eaf9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 0x7fb1e40c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1e979a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1e979a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60374/0 (socket says 172.21.15.53:60374) 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1e979a700 1 -- 172.21.15.53:0/4099547033 learned_addr learned my addr 172.21.15.53:0/4099547033 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c69b0 con 0x7fb1e40b0510 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c6c90 con 0x7fb1e406c790 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c6f70 con 0x7fb1e408a5a0 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7f39f97ff700 1 Processor -- start 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.883+0000 7f39f97ff700 1 -- start start 2023-11-28T11:17:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff138158c60 con 0x7ff138158890 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.884+0000 7fb1ea79c700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 0x7fb1e40c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e979a700 1 -- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 msgr2=0x7fb1e4089c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e979a700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e979a700 1 -- 172.21.15.53:0/4099547033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 msgr2=0x7fb1e40c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e979a700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 0x7fb1e40c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e979a700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00094a0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7f39f97ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39f40ce480 0x7f39f40c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1e9f9b700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7fb1ea79c700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 0x7fb1e40c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.886+0000 7fb1e979a700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1e001f040 tx=0x7fb1e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7f274359d700 1 -- 172.21.15.53:0/3151143968 shutdown_connections 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7f274359d700 1 -- 172.21.15.53:0/3151143968 wait complete. 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.885+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff13002d070 con 0x7ff138158890 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.887+0000 7ff12effd700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 0x7ff11806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.886+0000 7f39f2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39f40ce480 0x7f39f40c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.887+0000 7f39f2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39f40ce480 0x7f39f40c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40026/0 (socket says 172.21.15.53:40026) 2023-11-28T11:17:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.887+0000 7f39f27fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39f4161220 0x7f39f406b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46924/0 (socket says 172.21.15.53:46924) 2023-11-28T11:17:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.887+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff130086d40 con 0x7ff138158890 2023-11-28T11:17:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.887+0000 7ff13659c700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 0x7ff11806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.888+0000 7f274359d700 1 Processor -- start 2023-11-28T11:17:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.888+0000 7f39f2ffd700 1 -- 172.21.15.53:0/4251467357 learned_addr learned my addr 172.21.15.53:0/4251467357 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.888+0000 7ff13659c700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 0x7ff11806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.888+0000 7f274359d700 1 -- start start 2023-11-28T11:17:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.888+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f4162210 con 0x7f39f4161220 2023-11-28T11:17:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.889+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f41649d0 con 0x7f39f4164e60 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.889+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f406c020 con 0x7f39f40ce480 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 0x7f273c1635b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 0x7f273c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0b3d80 con 0x7f273c06b530 2023-11-28T11:17:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0c7720 con 0x7f273c160c00 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f274359d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0c8e90 con 0x7f273c0ad810 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f2742d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f2742d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38082/0 (socket says 172.21.15.53:38082) 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f2742d9c700 1 -- 172.21.15.53:0/2579458979 learned_addr learned my addr 172.21.15.53:0/2579458979 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730585316 0 0) 0x7f39f41649d0 con 0x7f39f4164e60 2023-11-28T11:17:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f2742d9c700 1 -- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 msgr2=0x7f273c0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.890+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39dc0035b0 con 0x7f39f4164e60 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2228979900 0 0) 0x7f39dc0035b0 con 0x7f39f4164e60 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 >> v2:172.21.15.53:3300/0 conn(0x7f39f40ce480 msgr2=0x7f39f40c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 --2- 172.21.15.53:0/4251467357 >> v2:172.21.15.53:3300/0 conn(0x7f39f40ce480 0x7f39f40c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 >> v1:172.21.15.53:6789/0 conn(0x7f39f4161220 legacy=0x7f39f406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f41616a0 con 0x7f39f4164e60 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f39e40016b0 con 0x7f39f4164e60 2023-11-28T11:17:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 >> v1:172.21.15.157:6789/0 conn(0x7f39f4164e60 legacy=0x7f39f40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc003a50 0x7f39dc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc004850 0x7f39dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 0x7f39dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f406c020 con 0x7f39dc005250 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc01d170 con 0x7f39dc004850 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc01d330 con 0x7f39dc003a50 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.891+0000 7f39f37fe700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 0x7f39dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0006af0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff1300535e0 con 0x7ff138158890 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 -- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc003a50 msgr2=0x7f39dc003e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc003a50 0x7f39dc003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 -- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc004850 msgr2=0x7f39dc004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc004850 0x7f39dc004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 -- 172.21.15.53:0/4251467357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc0035b0 con 0x7f39dc005250 2023-11-28T11:17:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f39f37fe700 1 -- 172.21.15.53:0/4251467357 reap_dead start 2023-11-28T11:17:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7f274259b700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 0x7f273c1635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2741d9a700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 0x7f273c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2742d9c700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 0x7f273c0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2742d9c700 1 -- 172.21.15.53:0/2579458979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 msgr2=0x7f273c1635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2742d9c700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 0x7f273c1635b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2742d9c700 1 -- 172.21.15.53:0/2579458979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c009550 con 0x7f273c160c00 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f274259b700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 0x7f273c1635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2742d9c700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f273400ef00 tx=0x7f273400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.892+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0003ce0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e001c680 con 0x7fb1e408a5a0 2023-11-28T11:17:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f39f37fe700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 0x7f39dc00ba10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c0e0 tx=0x7f39dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885624acfdad7866 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2741d9a700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 0x7f273c0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734010040 con 0x7f273c160c00 2023-11-28T11:17:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e416ddc0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.893+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e416dae0 con 0x7fb1e408a5a0 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400cc80 con 0x7f39dc005250 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e400cde0 con 0x7f39dc005250 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f39f1ffb700 1 -- 172.21.15.53:0/4251467357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4006aa0 con 0x7f39dc005250 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 msgr2=0x7f39dc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7f39f97ff700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 0x7f39dc00ba10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c0e0 tx=0x7f39dc005880 comp rx=0 tx=0).stop 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400bd30 con 0x7f273c160c00 2023-11-28T11:17:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734009ea0 con 0x7f273c160c00 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c164b30 con 0x7f273c160c00 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.894+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c164850 con 0x7f273c160c00 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40ca350 con 0x7fb1e408a5a0 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 shutdown_connections 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7f39f97ff700 1 --2- 172.21.15.53:0/4251467357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39dc005250 0x7f39dc00ba10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.895+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 >> 172.21.15.53:0/4251467357 conn(0x7f39f40cb670 msgr2=0x7f39f40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.897+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27240052f0 con 0x7f273c160c00 2023-11-28T11:17:56.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.897+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb1e0037070 con 0x7fb1e408a5a0 2023-11-28T11:17:56.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.898+0000 7fc5879ff700 1 Processor -- start 2023-11-28T11:17:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.899+0000 7fb1d67fc700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 0x7fb1c806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.899+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1e008f720 con 0x7fb1e408a5a0 2023-11-28T11:17:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.898+0000 7fc5879ff700 1 -- start start 2023-11-28T11:17:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.898+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2734014600 con 0x7f273c160c00 2023-11-28T11:17:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7f2732ffd700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 0x7f272006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2734086420 con 0x7f273c160c00 2023-11-28T11:17:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.899+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 shutdown_connections 2023-11-28T11:17:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.899+0000 7f39f97ff700 1 -- 172.21.15.53:0/4251467357 wait complete. 2023-11-28T11:17:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7fc5879ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5800d01b0 0x7fc5800d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7fc5879ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5800b16d0 0x7fc5800d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7fc5879ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5800b0230 con 0x7fc5800b20b0 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7fc5879ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58015cec0 con 0x7fc5800b16d0 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.900+0000 7fc5879ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5800b4570 con 0x7fc5800d01b0 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.901+0000 7fb1e9f9b700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 0x7fb1c806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.901+0000 7f39f97ff700 1 Processor -- start 2023-11-28T11:17:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.902+0000 7f39f97ff700 1 -- start start 2023-11-28T11:17:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.901+0000 7fc58579b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5800d01b0 0x7fc5800d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.902+0000 7fc58579b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5800d01b0 0x7fc5800d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38092/0 (socket says 172.21.15.53:38092) 2023-11-28T11:17:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.902+0000 7fc58579b700 1 -- 172.21.15.53:0/434730112 learned_addr learned my addr 172.21.15.53:0/434730112 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.902+0000 7fc585f9c700 1 --2- 172.21.15.53:0/434730112 >> v2:172.21.15.53:3300/0 conn(0x7fc5800b16d0 0x7fc5800d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f274259b700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 0x7f272006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.902+0000 7fb1e9f9b700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 0x7fb1c806a7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c280 tx=0x7fb1d800f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 0x7f39f4154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7fc58579b700 1 -- 172.21.15.53:0/434730112 >> v2:172.21.15.53:3300/0 conn(0x7fc5800b16d0 msgr2=0x7fc5800d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7fc58579b700 1 --2- 172.21.15.53:0/434730112 >> v2:172.21.15.53:3300/0 conn(0x7fc5800b16d0 0x7fc5800d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 0x7f39f4158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f274259b700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 0x7f272006a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2738009820 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f4158970 0x7f39f4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7fc58579b700 1 -- 172.21.15.53:0/434730112 >> v1:172.21.15.53:6789/0 conn(0x7fc5800b20b0 legacy=0x7fc5800b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7fc58579b700 1 -- 172.21.15.53:0/434730112 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800b4850 con 0x7fc5800d01b0 2023-11-28T11:17:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4070720 con 0x7f39f4158970 2023-11-28T11:17:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4158df0 con 0x7f39dc005250 2023-11-28T11:17:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f97ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f406fb30 con 0x7f39f40c5c30 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 0x7f39f4154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 0x7f39f4154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38096/0 (socket says 172.21.15.53:38096) 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f2ffd700 1 -- 172.21.15.53:0/91931147 learned_addr learned my addr 172.21.15.53:0/91931147 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.903+0000 7f39f27fc700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 0x7f39f4158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 -- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 msgr2=0x7f39f4154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 0x7f39f4154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 -- 172.21.15.53:0/91931147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f4158970 msgr2=0x7f39f4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f4158970 0x7f39f4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 -- 172.21.15.53:0/91931147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4161c50 con 0x7f39f40c5c30 2023-11-28T11:17:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.904+0000 7f39f27fc700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 0x7f39f4158430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f39f40690f0 tx=0x7f39e800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7fc58579b700 1 --2- 172.21.15.53:0/434730112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800d01b0 0x7fc5800d0c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5800b5bb0 tx=0x7fc578009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b271fbde68d86e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8005cb0 con 0x7f39f40c5c30 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f4161970 con 0x7f39f40c5c30 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8005e10 con 0x7f39f40c5c30 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f4161f30 con 0x7f39f40c5c30 2023-11-28T11:17:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.905+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8005cb0 con 0x7f39f40c5c30 2023-11-28T11:17:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2734052c10 con 0x7f273c160c00 2023-11-28T11:17:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1e005bf10 con 0x7fb1e408a5a0 2023-11-28T11:17:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7fc577fff700 1 -- 172.21.15.53:0/434730112 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578004230 con 0x7fc5800d01b0 2023-11-28T11:17:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7fc577fff700 1 -- 172.21.15.53:0/434730112 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578004390 con 0x7fc5800d01b0 2023-11-28T11:17:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7fc577fff700 1 -- 172.21.15.53:0/434730112 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5780036a0 con 0x7fc5800d01b0 2023-11-28T11:17:56.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.906+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40052f0 con 0x7f39f40c5c30 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.907+0000 7fc5879ff700 1 -- 172.21.15.53:0/434730112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800d01b0 msgr2=0x7fc5800d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fc5879ff700 1 --2- 172.21.15.53:0/434730112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800d01b0 0x7fc5800d0c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5800b5bb0 tx=0x7fc578009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fc5879ff700 1 -- 172.21.15.53:0/434730112 shutdown_connections 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fc5879ff700 1 --2- 172.21.15.53:0/434730112 >> v2:172.21.15.53:3300/0 conn(0x7fc5800b16d0 0x7fc5800d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fc5879ff700 1 --2- 172.21.15.53:0/434730112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800d01b0 0x7fc5800d0c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.909+0000 7fc5879ff700 1 -- 172.21.15.53:0/434730112 >> 172.21.15.53:0/434730112 conn(0x7fc58009c3f0 msgr2=0x7fc5800d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.909+0000 7fc5879ff700 1 -- 172.21.15.53:0/434730112 shutdown_connections 2023-11-28T11:17:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.909+0000 7fc5879ff700 1 -- 172.21.15.53:0/434730112 wait complete. 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fa544dff700 1 Processor -- start 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.908+0000 7fa544dff700 1 -- start start 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7fa544dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5400c6030 0x7fa5400cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7fa544dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5400c6580 0x7fa5400b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7fa544dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400ce400 con 0x7fa5400d0c20 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7fa544dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5400cf080 con 0x7fa5400c6030 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7fa544dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5400b49e0 con 0x7fa5400c6580 2023-11-28T11:17:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.911+0000 7fa53e59c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa5400d0c20 0x7fa5400d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43608/0 (socket says 172.21.15.53:43608) 2023-11-28T11:17:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.911+0000 7fa53e59c700 1 -- 172.21.15.53:0/3244240821 learned_addr learned my addr 172.21.15.53:0/3244240821 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.910+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39e8019030 con 0x7f39f40c5c30 2023-11-28T11:17:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.911+0000 7f39db7fe700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 0x7f39e006a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.911+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335153280 0 0) 0x7fa5400ce400 con 0x7fa5400d0c20 2023-11-28T11:17:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.911+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39e8084890 con 0x7f39f40c5c30 2023-11-28T11:17:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.912+0000 7f39f2ffd700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 0x7f39e006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7f39f2ffd700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 0x7f39e006a860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39f40c8dc0 tx=0x7f39ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.912+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5280035b0 con 0x7fa5400d0c20 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.912+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3244240821 >> v2:172.21.15.53:3300/0 conn(0x7fa5400c6030 0x7fa5400cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3244240821 >> v2:172.21.15.157:3300/0 conn(0x7fa5400c6580 0x7fa5400b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2122690668 0 0) 0x7fa5280035b0 con 0x7fa5400d0c20 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 >> v2:172.21.15.157:3300/0 conn(0x7fa5400c6580 msgr2=0x7fa5400b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 --2- 172.21.15.53:0/3244240821 >> v2:172.21.15.157:3300/0 conn(0x7fa5400c6580 0x7fa5400b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 >> v2:172.21.15.53:3300/0 conn(0x7fa5400c6030 msgr2=0x7fa5400cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 --2- 172.21.15.53:0/3244240821 >> v2:172.21.15.53:3300/0 conn(0x7fa5400c6030 0x7fa5400cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400ce6e0 con 0x7fa5400d0c20 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa5380019f0 con 0x7fa5400d0c20 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 >> v1:172.21.15.157:6789/0 conn(0x7fa5400d0c20 legacy=0x7fa5400d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.913+0000 7fa53d59a700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53d59a700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa528004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53d59a700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528005200 0x7fa52800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400cf080 con 0x7fa528005200 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400b49e0 con 0x7fa528003a00 2023-11-28T11:17:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52800c160 con 0x7fa528004850 2023-11-28T11:17:56.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fc5879ff700 1 Processor -- start 2023-11-28T11:17:56.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fc5879ff700 1 -- start start 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.914+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa528004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.915+0000 7fa53e59c700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.915+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528005200 0x7fa52800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53dd9b700 1 -- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa528003a00 msgr2=0x7fa528003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53dd9b700 1 -- 172.21.15.53:0/3244240821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528005200 msgr2=0x7fa52800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528005200 0x7fa52800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53dd9b700 1 -- 172.21.15.53:0/3244240821 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400ce6e0 con 0x7fa528004850 2023-11-28T11:17:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528005200 0x7fa52800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fa53ed9d700 1 -- 172.21.15.53:0/3244240821 reap_dead start 2023-11-28T11:17:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc5879ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5800b16d0 0x7fc5800b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc5879ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc58579b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5800b16d0 0x7fc5800b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc5879ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 0x7fc5800b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc5879ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58014d170 con 0x7fc5800b16d0 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.916+0000 7fc5879ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5800c83b0 con 0x7fc5800b20b0 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fc5879ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5800b8cb0 con 0x7fc5800d01b0 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39e804b5e0 con 0x7f39f40c5c30 2023-11-28T11:17:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fc584f9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fc584f9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38116/0 (socket says 172.21.15.53:38116) 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fc584f9a700 1 -- 172.21.15.53:0/1218946670 learned_addr learned my addr 172.21.15.53:0/1218946670 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fc585f9c700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 0x7fc5800b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc584f9a700 1 -- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 msgr2=0x7fc5800b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc584f9a700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 0x7fc5800b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc584f9a700 1 -- 172.21.15.53:0/1218946670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5800b16d0 msgr2=0x7fc5800b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc584f9a700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5800b16d0 0x7fc5800b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc584f9a700 1 -- 172.21.15.53:0/1218946670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc578009550 con 0x7fc5800b20b0 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fc585f9c700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 0x7fc5800b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.919+0000 7fc584f9a700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c500 tx=0x7fc57000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.919+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570011790 con 0x7fc5800b20b0 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.919+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5700118f0 con 0x7fc5800b20b0 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa528004cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa534009890 tx=0x7fa528005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd078e77c21a09e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400c070 con 0x7fa528004850 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa534013610 con 0x7fa528004850 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.917+0000 7fa53d59a700 1 -- 172.21.15.53:0/3244240821 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340139a0 con 0x7fa528004850 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.919+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570011ca0 con 0x7fc5800b20b0 2023-11-28T11:17:56.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.919+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5800afec0 con 0x7fc5800b20b0 2023-11-28T11:17:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fa544dff700 1 -- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 msgr2=0x7fa528004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.918+0000 7fa544dff700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa528004cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa534009890 tx=0x7fa528005880 comp rx=0 tx=0).stop 2023-11-28T11:17:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.920+0000 7fa544dff700 1 -- 172.21.15.53:0/3244240821 shutdown_connections 2023-11-28T11:17:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.920+0000 7fa544dff700 1 --2- 172.21.15.53:0/3244240821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa528004cc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.920+0000 7fa544dff700 1 -- 172.21.15.53:0/3244240821 >> 172.21.15.53:0/3244240821 conn(0x7fa54009c3f0 msgr2=0x7fa5400d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.921+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5800afbe0 con 0x7fc5800b20b0 2023-11-28T11:17:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.923+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc570010050 con 0x7fc5800b20b0 2023-11-28T11:17:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.923+0000 7fa544dff700 1 -- 172.21.15.53:0/3244240821 shutdown_connections 2023-11-28T11:17:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.923+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5800b2600 con 0x7fc5800b20b0 2023-11-28T11:17:56.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.925+0000 7fc575ffb700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 0x7fc56c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.926+0000 7fc58579b700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 0x7fc56c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.923+0000 7fa544dff700 1 -- 172.21.15.53:0/3244240821 wait complete. 2023-11-28T11:17:56.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.926+0000 7fa544dff700 1 Processor -- start 2023-11-28T11:17:56.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.927+0000 7fc58579b700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 0x7fc56c06a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc580070c10 tx=0x7fc578008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.927+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc57005d5d0 con 0x7fc5800b20b0 2023-11-28T11:17:56.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.927+0000 7fa544dff700 1 -- start start 2023-11-28T11:17:56.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.929+0000 7fa544dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa54015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa544dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa544dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 0x7fa54015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa54015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa54015e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60424/0 (socket says 172.21.15.53:60424) 2023-11-28T11:17:56.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa53dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa53dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40100/0 (socket says 172.21.15.53:40100) 2023-11-28T11:17:56.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa53e59c700 1 -- 172.21.15.53:0/3212429994 learned_addr learned my addr 172.21.15.53:0/3212429994 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 0x7fa54015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.930+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401f9c10 con 0x7fa5401fa620 2023-11-28T11:17:56.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401fa190 con 0x7fa5401f5e60 2023-11-28T11:17:56.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54015ec80 con 0x7fa528004850 2023-11-28T11:17:56.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.932+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc57008a060 con 0x7fc5800b20b0 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 -- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 msgr2=0x7fa54015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa54015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 -- 172.21.15.53:0/3212429994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 msgr2=0x7fa54015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53ed9d700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400ceca0 con 0x7fa5401f5e60 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.931+0000 7fa53dd9b700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.932+0000 7fa53ed9d700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 0x7fa54015c770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa54006d430 tx=0x7fa530010750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.933+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530004230 con 0x7fa5401f5e60 2023-11-28T11:17:56.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.933+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53000c050 con 0x7fa5401f5e60 2023-11-28T11:17:56.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.933+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300036a0 con 0x7fa5401f5e60 2023-11-28T11:17:56.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.933+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5400ce9c0 con 0x7fa5401f5e60 2023-11-28T11:17:56.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.933+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5400ce0b0 con 0x7fa5401f5e60 2023-11-28T11:17:56.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.937+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa530012040 con 0x7fa5401f5e60 2023-11-28T11:17:56.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.939+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5240052f0 con 0x7fa5401f5e60 2023-11-28T11:17:56.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.940+0000 7fa52effd700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 0x7fa52006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.941+0000 7fa53e59c700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 0x7fa52006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.941+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa530085520 con 0x7fa5401f5e60 2023-11-28T11:17:56.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.941+0000 7fa53e59c700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 0x7fa52006a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa538001b60 tx=0x7fa538006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.946+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa530051c70 con 0x7fa5401f5e60 2023-11-28T11:17:56.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.947+0000 7f766d5ff700 1 Processor -- start 2023-11-28T11:17:56.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.947+0000 7f766d5ff700 1 -- start start 2023-11-28T11:17:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.949+0000 7f766d5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.949+0000 7f766d5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 0x7f7668160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.949+0000 7f766d5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76680d2480 con 0x7f7668160cc0 2023-11-28T11:17:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.949+0000 7f766d5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76681602f0 con 0x7f76680cd500 2023-11-28T11:17:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.949+0000 7f766d5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766815d940 con 0x7f7668156d10 2023-11-28T11:17:56.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.950+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.950+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60428/0 (socket says 172.21.15.53:60428) 2023-11-28T11:17:56.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.950+0000 7f7666ffd700 1 -- 172.21.15.53:0/385780482 learned_addr learned my addr 172.21.15.53:0/385780482 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.951+0000 7f76667fc700 1 --2- 172.21.15.53:0/385780482 >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 0x7f7668160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.951+0000 7f7666ffd700 1 -- 172.21.15.53:0/385780482 >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 msgr2=0x7f7668160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.951+0000 7f7666ffd700 1 --2- 172.21.15.53:0/385780482 >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 0x7f7668160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.951+0000 7f7666ffd700 1 -- 172.21.15.53:0/385780482 >> v1:172.21.15.53:6789/0 conn(0x7f7668160cc0 legacy=0x7f76680cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.952+0000 7f7665ffb700 1 -- 172.21.15.53:0/385780482 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575409708 0 0) 0x7f76680d2480 con 0x7f7668160cc0 2023-11-28T11:17:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.952+0000 7f7666ffd700 1 -- 172.21.15.53:0/385780482 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76680d1850 con 0x7f7668156d10 2023-11-28T11:17:56.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.952+0000 7f76667fc700 1 --2- 172.21.15.53:0/385780482 >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 0x7f7668160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.953+0000 7f7666ffd700 1 --2- 172.21.15.53:0/385780482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7660007d70 tx=0x7f766000c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6450195dfcf72b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.953+0000 7f7665ffb700 1 -- 172.21.15.53:0/385780482 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600113f0 con 0x7f7668156d10 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.953+0000 7f7665ffb700 1 -- 172.21.15.53:0/385780482 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660011550 con 0x7f7668156d10 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.953+0000 7f7665ffb700 1 -- 172.21.15.53:0/385780482 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660011900 con 0x7f7668156d10 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.954+0000 7f766d5ff700 1 -- 172.21.15.53:0/385780482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 msgr2=0x7f766815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.954+0000 7f766d5ff700 1 --2- 172.21.15.53:0/385780482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7660007d70 tx=0x7f766000c430 comp rx=0 tx=0).stop 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 -- 172.21.15.53:0/385780482 shutdown_connections 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 --2- 172.21.15.53:0/385780482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7668156d10 0x7f766815d400 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 --2- 172.21.15.53:0/385780482 >> v2:172.21.15.53:3300/0 conn(0x7f76680cd500 0x7f7668160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 -- 172.21.15.53:0/385780482 >> 172.21.15.53:0/385780482 conn(0x7f766806c420 msgr2=0x7f766806c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 -- 172.21.15.53:0/385780482 shutdown_connections 2023-11-28T11:17:56.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.956+0000 7f766d5ff700 1 -- 172.21.15.53:0/385780482 wait complete. 2023-11-28T11:17:56.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.959+0000 7f766d5ff700 1 Processor -- start 2023-11-28T11:17:56.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.960+0000 7ff737fff700 1 Processor -- start 2023-11-28T11:17:56.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.960+0000 7ff737fff700 1 -- start start 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.959+0000 7f766d5ff700 1 -- start start 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7668160cc0 0x7f76680cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76681570e0 con 0x7f7668156d10 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76680cbbc0 con 0x7f7668160cc0 2023-11-28T11:17:56.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.961+0000 7f766d5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76680cbea0 con 0x7f76680cd500 2023-11-28T11:17:56.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff737fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73815a8c0 0x7ff7380cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f76667fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f76667fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40122/0 (socket says 172.21.15.53:40122) 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60438/0 (socket says 172.21.15.53:60438) 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f7666ffd700 1 -- 172.21.15.53:0/233921022 learned_addr learned my addr 172.21.15.53:0/233921022 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff737fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff73806f790 0x7ff73806fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff737fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff738070120 con 0x7ff7380b60d0 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff737fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380cc840 con 0x7ff73806f790 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff737fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d0d30 con 0x7ff73815a8c0 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff736ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73815a8c0 0x7ff7380cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff736ffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73815a8c0 0x7ff7380cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60440/0 (socket says 172.21.15.53:60440) 2023-11-28T11:17:56.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7ff736ffd700 1 -- 172.21.15.53:0/3133050548 learned_addr learned my addr 172.21.15.53:0/3133050548 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.962+0000 7f76677fe700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7668160cc0 0x7f76680cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7f7666ffd700 1 -- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7668160cc0 msgr2=0x7f76680cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7f7666ffd700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7668160cc0 0x7f76680cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> v2:172.21.15.53:3300/0 conn(0x7ff73806f790 0x7ff73806fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042134125 0 0) 0x7ff730001490 con 0x7ff7380b60d0 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7200035b0 con 0x7ff7380b60d0 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2491644778 0 0) 0x7ff7200035b0 con 0x7ff7380b60d0 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73815a8c0 msgr2=0x7ff7380cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73815a8c0 0x7ff7380cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 >> v2:172.21.15.53:3300/0 conn(0x7ff73806f790 msgr2=0x7ff73806fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 --2- 172.21.15.53:0/3133050548 >> v2:172.21.15.53:3300/0 conn(0x7ff73806f790 0x7ff73806fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380d1010 con 0x7ff7380b60d0 2023-11-28T11:17:56.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7f7666ffd700 1 -- 172.21.15.53:0/233921022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 msgr2=0x7f76680cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.963+0000 7f7666ffd700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7ff730001b80 con 0x7ff7380b60d0 2023-11-28T11:17:56.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7f7666ffd700 1 -- 172.21.15.53:0/233921022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600079d0 con 0x7f76680cd500 2023-11-28T11:17:56.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7f76667fc700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 >> v1:172.21.15.157:6789/0 conn(0x7ff7380b60d0 legacy=0x7ff7380b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff720003a00 0x7ff720003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004850 0x7ff720004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720005200 0x7ff72000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380cc840 con 0x7ff720003a00 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d0d30 con 0x7ff720004850 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72000c050 con 0x7ff720005200 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> v2:172.21.15.53:3300/0 conn(0x7ff73806f790 0x7ff73806fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004850 0x7ff720004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff736ffd700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff720003a00 0x7ff720003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 -- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720005200 msgr2=0x7ff72000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720005200 0x7ff72000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 -- 172.21.15.53:0/3133050548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff720003a00 msgr2=0x7ff720003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff720003a00 0x7ff720003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 -- 172.21.15.53:0/3133050548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200035b0 con 0x7ff720004850 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.964+0000 7f7666ffd700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7660014040 tx=0x7f7660002350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766001f070 con 0x7f76680cd500 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660007ec0 con 0x7f76680cd500 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7f766d5ff700 1 -- 172.21.15.53:0/233921022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76680d29b0 con 0x7f76680cd500 2023-11-28T11:17:56.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600113f0 con 0x7f76680cd500 2023-11-28T11:17:56.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7f766d5ff700 1 -- 172.21.15.53:0/233921022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76680d26d0 con 0x7f76680cd500 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.965+0000 7ff7367fc700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004850 0x7ff720004cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00db60 tx=0x7ff720005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893bb68a39adba33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.966+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00bb40 con 0x7ff720004850 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.966+0000 7ff735ffb700 1 -- 172.21.15.53:0/3133050548 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c013040 con 0x7ff720004850 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.967+0000 7ff737fff700 1 -- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004850 msgr2=0x7ff720004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.967+0000 7ff737fff700 1 --2- 172.21.15.53:0/3133050548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004850 0x7ff720004cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00db60 tx=0x7ff720005880 comp rx=0 tx=0).stop 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.968+0000 7ff7377fe700 1 -- 172.21.15.53:0/3133050548 reap_dead start 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.968+0000 7ff737fff700 1 -- 172.21.15.53:0/3133050548 shutdown_connections 2023-11-28T11:17:56.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.968+0000 7ff737fff700 1 -- 172.21.15.53:0/3133050548 >> 172.21.15.53:0/3133050548 conn(0x7ff73806af00 msgr2=0x7ff73806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:56.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.968+0000 7ff737fff700 1 -- 172.21.15.53:0/3133050548 shutdown_connections 2023-11-28T11:17:56.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.969+0000 7ff737fff700 1 -- 172.21.15.53:0/3133050548 wait complete. 2023-11-28T11:17:56.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.970+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76680cf810 con 0x7f76680cd500 2023-11-28T11:17:56.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.970+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f766001a420 con 0x7f76680cd500 2023-11-28T11:17:56.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.971+0000 7f764f7fe700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 0x7f765406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.971+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7660087670 con 0x7f76680cd500 2023-11-28T11:17:56.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.972+0000 7ff737fff700 1 Processor -- start 2023-11-28T11:17:56.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.972+0000 7f76667fc700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 0x7f765406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.972+0000 7ff737fff700 1 -- start start 2023-11-28T11:17:56.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.973+0000 7f76667fc700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 0x7f765406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76681566c0 tx=0x7f7658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.974+0000 7ff737fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 0x7ff7381f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.975+0000 7ff737fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.975+0000 7ff737fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.975+0000 7ff7367fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.975+0000 7ff7377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff7377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40150/0 (socket says 172.21.15.53:40150) 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff7367fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38158/0 (socket says 172.21.15.53:38158) 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff7377fe700 1 -- 172.21.15.53:0/994729182 learned_addr learned my addr 172.21.15.53:0/994729182 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d0060 con 0x7ff7380b27f0 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c7ee0 con 0x7ff7380cf6b0 2023-11-28T11:17:56.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c81c0 con 0x7ff720004850 2023-11-28T11:17:56.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.976+0000 7ff736ffd700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 0x7ff7381f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7367fc700 1 -- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 msgr2=0x7ff7381f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7367fc700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 0x7ff7381f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7367fc700 1 -- 172.21.15.53:0/994729182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 msgr2=0x7ff7380b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7367fc700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7367fc700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380d15d0 con 0x7ff7380cf6b0 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.977+0000 7ff7377fe700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff736ffd700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 0x7ff7381f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:56.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff7367fc700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00db30 tx=0x7ff72c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7660053d80 con 0x7f76680cd500 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00bb40 con 0x7ff7380cf6b0 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00d590 con 0x7ff7380cf6b0 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0226a0 con 0x7ff7380cf6b0 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.978+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7380d12f0 con 0x7ff7380cf6b0 2023-11-28T11:17:56.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.979+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380d18b0 con 0x7ff7380cf6b0 2023-11-28T11:17:56.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.982+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff72c029030 con 0x7ff7380cf6b0 2023-11-28T11:17:56.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.982+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff738061e80 con 0x7ff7380cf6b0 2023-11-28T11:17:56.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.983+0000 7ff71f7fe700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 0x7ff72406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:56.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.983+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff72c086a80 con 0x7ff7380cf6b0 2023-11-28T11:17:56.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.984+0000 7ff736ffd700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 0x7ff72406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:56.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.986+0000 7ff736ffd700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 0x7ff72406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7380c7cf0 tx=0x7ff728008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:56.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:56.990+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff72c0531f0 con 0x7ff7380cf6b0 2023-11-28T11:17:57.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.359+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb1e40a8dc0 con 0x7fb1e408a5a0 2023-11-28T11:17:57.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.359+0000 7fb1d67fc700 1 -- 172.21.15.53:0/4099547033 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1e005f530 con 0x7fb1e408a5a0 2023-11-28T11:17:57.360 INFO:teuthology.orchestra.run.smithi053.stdout:60129542148 2023-11-28T11:17:57.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.362+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff1380745c0 con 0x7ff138158890 2023-11-28T11:17:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.363+0000 7ff12effd700 1 -- 172.21.15.53:0/3527849348 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff13002b090 con 0x7ff138158890 2023-11-28T11:17:57.365 INFO:teuthology.orchestra.run.smithi053.stdout:60129542147 2023-11-28T11:17:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 msgr2=0x7fb1c806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 0x7fb1c806a7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c280 tx=0x7fb1d800f040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 msgr2=0x7fb1e40c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1e001f040 tx=0x7fb1e0004750 comp rx=0 tx=0).stop 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 shutdown_connections 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1c8068340 0x7fb1c806a7f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1e40b0510 0x7fb1e40c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1e406c790 0x7fb1e4089c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 --2- 172.21.15.53:0/4099547033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e408a5a0 0x7fb1e40c7ee0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 >> 172.21.15.53:0/4099547033 conn(0x7fb1e40cb510 msgr2=0x7fb1e40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 shutdown_connections 2023-11-28T11:17:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.364+0000 7fb1eaf9d700 1 -- 172.21.15.53:0/4099547033 wait complete. 2023-11-28T11:17:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.365+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f27240055a0 con 0x7f273c160c00 2023-11-28T11:17:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.366+0000 7f2732ffd700 1 -- 172.21.15.53:0/2579458979 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2734056230 con 0x7f273c160c00 2023-11-28T11:17:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:60129542148 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 msgr2=0x7ff11806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 0x7ff11806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007f30 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 msgr2=0x7ff1381f1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 0x7ff1381f1400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff13001a000 tx=0x7ff130004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 shutdown_connections 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1180683c0 0x7ff11806a870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff138158890 0x7ff1381f1400 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13806d4f0 0x7ff138159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 --2- 172.21.15.53:0/3527849348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1380b0bb0 0x7ff13815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.367+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 >> 172.21.15.53:0/3527849348 conn(0x7ff1380cb670 msgr2=0x7ff1380ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.368+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 shutdown_connections 2023-11-28T11:17:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.368+0000 7ff13cbff700 1 -- 172.21.15.53:0/3527849348 wait complete. 2023-11-28T11:17:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 msgr2=0x7f272006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 0x7f272006a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2738009820 tx=0x7f2738008040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 msgr2=0x7f273c0c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f273400ef00 tx=0x7f273400b390 comp rx=0 tx=0).stop 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 shutdown_connections 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2720068340 0x7f272006a7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c160c00 0x7f273c0c7080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f273c06b530 0x7f273c1635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 --2- 172.21.15.53:0/2579458979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273c0ad810 0x7f273c0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.374+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 >> 172.21.15.53:0/2579458979 conn(0x7f273c0cb510 msgr2=0x7f273c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.375+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 shutdown_connections 2023-11-28T11:17:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.375+0000 7f274359d700 1 -- 172.21.15.53:0/2579458979 wait complete. 2023-11-28T11:17:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.379+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39d40055a0 con 0x7f39f40c5c30 2023-11-28T11:17:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.383+0000 7f39db7fe700 1 -- 172.21.15.53:0/91931147 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39e8017090 con 0x7f39f40c5c30 2023-11-28T11:17:57.388 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:57.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 msgr2=0x7f39e006a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 0x7f39e006a860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39f40c8dc0 tx=0x7f39ec00b040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 msgr2=0x7f39f4158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 0x7f39f4158430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f39f40690f0 tx=0x7f39e800c6e0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 shutdown_connections 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39e00683b0 0x7f39e006a860 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39f40c8dc0 tx=0x7f39ec00b040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f4158970 0x7f39f4070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc005250 0x7f39f4154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 --2- 172.21.15.53:0/91931147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f40c5c30 0x7f39f4158430 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.389+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 >> 172.21.15.53:0/91931147 conn(0x7f39f40cb670 msgr2=0x7f39f415f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.391+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 shutdown_connections 2023-11-28T11:17:57.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.391+0000 7f39f97ff700 1 -- 172.21.15.53:0/91931147 wait complete. 2023-11-28T11:17:57.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.393+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc5800c8210 con 0x7fc5800b20b0 2023-11-28T11:17:57.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.394+0000 7fc575ffb700 1 -- 172.21.15.53:0/1218946670 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc5800c8210 con 0x7fc5800b20b0 2023-11-28T11:17:57.395 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:57.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.4 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 msgr2=0x7fc56c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 0x7fc56c06a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc580070c10 tx=0x7fc578008030 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 msgr2=0x7fc5800c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c500 tx=0x7fc57000c810 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 shutdown_connections 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc56c068340 0x7fc56c06a7f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5800d01b0 0x7fc5800b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5800b16d0 0x7fc5800b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 --2- 172.21.15.53:0/1218946670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5800b20b0 0x7fc5800c8b70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 >> 172.21.15.53:0/1218946670 conn(0x7fc58009c3f0 msgr2=0x7fc58014f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 shutdown_connections 2023-11-28T11:17:57.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.399+0000 7fc5879ff700 1 -- 172.21.15.53:0/1218946670 wait complete. 2023-11-28T11:17:57.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.6 2023-11-28T11:17:57.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2023-11-28T11:17:57.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.413+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5240055a0 con 0x7fa5401f5e60 2023-11-28T11:17:57.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.414+0000 7fa52effd700 1 -- 172.21.15.53:0/3212429994 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa530055290 con 0x7fa5401f5e60 2023-11-28T11:17:57.415 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:57.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 msgr2=0x7fa52006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 0x7fa52006a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa538001b60 tx=0x7fa538006040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 msgr2=0x7fa54015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 0x7fa54015c770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa54006d430 tx=0x7fa530010750 comp rx=0 tx=0).stop 2023-11-28T11:17:57.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 shutdown_connections 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa5200682f0 0x7fa52006a7a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa528004850 0x7fa54015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5401f5e60 0x7fa54015c770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 --2- 172.21.15.53:0/3212429994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5401fa620 0x7fa54015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 >> 172.21.15.53:0/3212429994 conn(0x7fa54009c3f0 msgr2=0x7fa5400b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 shutdown_connections 2023-11-28T11:17:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.418+0000 7fa544dff700 1 -- 172.21.15.53:0/3212429994 wait complete. 2023-11-28T11:17:57.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.1 2023-11-28T11:17:57.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2023-11-28T11:17:57.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.429+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7668067d90 con 0x7f76680cd500 2023-11-28T11:17:57.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.430+0000 7f764f7fe700 1 -- 172.21.15.53:0/233921022 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f766002a020 con 0x7f76680cd500 2023-11-28T11:17:57.431 INFO:teuthology.orchestra.run.smithi053.stdout:60129542147 2023-11-28T11:17:57.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.430+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff73806eb70 con 0x7ff7380cf6b0 2023-11-28T11:17:57.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.431+0000 7ff71f7fe700 1 -- 172.21.15.53:0/994729182 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff72c027020 con 0x7ff7380cf6b0 2023-11-28T11:17:57.432 INFO:teuthology.orchestra.run.smithi053.stdout:60129542148 2023-11-28T11:17:57.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 msgr2=0x7f765406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 0x7f765406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76681566c0 tx=0x7f7658008040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 msgr2=0x7f76682019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7660014040 tx=0x7f7660002350 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 shutdown_connections 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f76540682f0 0x7f765406a7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76680cd500 0x7f76682019e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7668160cc0 0x7f76680cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 --2- 172.21.15.53:0/233921022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7668156d10 0x7f76680cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.432+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 >> 172.21.15.53:0/233921022 conn(0x7f766806c420 msgr2=0x7f76680d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.433+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 shutdown_connections 2023-11-28T11:17:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.433+0000 7f764d7fa700 1 -- 172.21.15.53:0/233921022 wait complete. 2023-11-28T11:17:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 msgr2=0x7ff72406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 0x7ff72406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7380c7cf0 tx=0x7ff728008040 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 msgr2=0x7ff7380cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00db30 tx=0x7ff72c004750 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 shutdown_connections 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7240682f0 0x7ff72406a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff720004850 0x7ff7381f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b27f0 0x7ff7380b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 --2- 172.21.15.53:0/994729182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7380cf6b0 0x7ff7380cfb20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 >> 172.21.15.53:0/994729182 conn(0x7ff73806af00 msgr2=0x7ff73806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 shutdown_connections 2023-11-28T11:17:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:57.434+0000 7ff737fff700 1 -- 172.21.15.53:0/994729182 wait complete. 2023-11-28T11:17:57.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.2 2023-11-28T11:17:57.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.7 2023-11-28T11:17:57.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.3 2023-11-28T11:17:58.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:17:58.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:17:58.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:17:58.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:17:58.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:17:58.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:17:58.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:17:58.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:17:58.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.729+0000 7f1d08bff700 1 Processor -- start 2023-11-28T11:17:58.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.729+0000 7f1d08bff700 1 -- start start 2023-11-28T11:17:58.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.731+0000 7f1d08bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d04070210 0x7f1d041676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.731+0000 7f1d08bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040ccbc0 con 0x7f1d0415c970 2023-11-28T11:17:58.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.731+0000 7f1d08bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040ce030 con 0x7f1d040c2260 2023-11-28T11:17:58.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.731+0000 7f1d08bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0415cd40 con 0x7f1d04070210 2023-11-28T11:17:58.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.730+0000 7f67d27ff700 1 Processor -- start 2023-11-28T11:17:58.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.731+0000 7f67d27ff700 1 -- start start 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 0x7f67cc0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67cc160f00 0x7f67cc164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67cc164d80 con 0x7f67cc160b30 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67cc06ba50 con 0x7f67cc160f00 2023-11-28T11:17:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67d27ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc164f90 con 0x7f67cc0cc730 2023-11-28T11:17:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f1d01d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1d040c2260 0x7f1d0406fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43628/0 (socket says 172.21.15.53:43628) 2023-11-28T11:17:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f1d01d9b700 1 -- 172.21.15.53:0/3630070743 learned_addr learned my addr 172.21.15.53:0/3630070743 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f0d5bfff700 1 Processor -- start 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.733+0000 7f67cb7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67cc160f00 0x7f67cc164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67cc160f00 0x7f67cc164840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38170/0 (socket says 172.21.15.53:38170) 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 -- 172.21.15.53:0/2729702599 learned_addr learned my addr 172.21.15.53:0/2729702599 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67d0d9c700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cbfff700 1 --2- 172.21.15.53:0/2729702599 >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 0x7f67cc0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 -- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 msgr2=0x7f67cc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 -- 172.21.15.53:0/2729702599 >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 msgr2=0x7f67cc0c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 --2- 172.21.15.53:0/2729702599 >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 0x7f67cc0c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cb7fe700 1 -- 172.21.15.53:0/2729702599 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc165270 con 0x7f67cc160f00 2023-11-28T11:17:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f67cbfff700 1 --2- 172.21.15.53:0/2729702599 >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 0x7f67cc0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f67cb7fe700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160f00 0x7f67cc164840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0c9be0 tx=0x7f67bc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a47d23b9958386d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f0d5bfff700 1 -- start start 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.734+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568562862 0 0) 0x7f1d040ce030 con 0x7f1d040c2260 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec0035b0 con 0x7f1d040c2260 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889041231 0 0) 0x7f1d040ccbc0 con 0x7f1d0415c970 2023-11-28T11:17:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d040ce030 con 0x7f1d0415c970 2023-11-28T11:17:58.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.735+0000 7f1d02d9d700 1 --2- 172.21.15.53:0/3630070743 >> v2:172.21.15.53:3300/0 conn(0x7f1d04070210 0x7f1d041676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f67caffd700 1 -- 172.21.15.53:0/2729702599 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc011830 con 0x7f67cc160f00 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f67caffd700 1 -- 172.21.15.53:0/2729702599 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc011990 con 0x7f67cc160f00 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f67caffd700 1 -- 172.21.15.53:0/2729702599 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc011d20 con 0x7f67cc160f00 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 -- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160f00 msgr2=0x7f67cc164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160f00 0x7f67cc164840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0c9be0 tx=0x7f67bc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 -- 172.21.15.53:0/2729702599 shutdown_connections 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 --2- 172.21.15.53:0/2729702599 >> v2:172.21.15.53:3300/0 conn(0x7f67cc160b30 0x7f67cc0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 --2- 172.21.15.53:0/2729702599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160f00 0x7f67cc164840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f67d27ff700 1 -- 172.21.15.53:0/2729702599 >> 172.21.15.53:0/2729702599 conn(0x7f67cc0cb670 msgr2=0x7f67cc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350374447 0 0) 0x7f1d040ce030 con 0x7f1d0415c970 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 >> v2:172.21.15.53:3300/0 conn(0x7f1d04070210 msgr2=0x7f1d041676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 --2- 172.21.15.53:0/3630070743 >> v2:172.21.15.53:3300/0 conn(0x7f1d04070210 0x7f1d041676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 >> v1:172.21.15.157:6789/0 conn(0x7f1d040c2260 legacy=0x7f1d0406fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0406cc90 con 0x7f1d0415c970 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f1cf4001670 con 0x7f1d0415c970 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 >> v1:172.21.15.53:6789/0 conn(0x7f1d0415c970 legacy=0x7f1d0406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 0x7f1cec004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cec0037f0 0x7f1cec003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec0041a0 0x7f1cec014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0415cd40 con 0x7f1cec0037f0 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec014870 con 0x7f1cec004bf0 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.736+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec014a30 con 0x7f1cec0041a0 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f1d0259c700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 0x7f1cec004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.737+0000 7f1d01d9b700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cec0037f0 0x7f1cec003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d0259c700 1 -- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec0041a0 msgr2=0x7f1cec014220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d0259c700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec0041a0 0x7f1cec014220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d0259c700 1 -- 172.21.15.53:0/3630070743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cec0037f0 msgr2=0x7f1cec003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d0259c700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cec0037f0 0x7f1cec003c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d0259c700 1 -- 172.21.15.53:0/3630070743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0406cc90 con 0x7f1cec004bf0 2023-11-28T11:17:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f0d5bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d5c0ccbe0 0x7f0d5c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f67d27ff700 1 -- 172.21.15.53:0/2729702599 shutdown_connections 2023-11-28T11:17:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f1d02d9d700 1 -- 172.21.15.53:0/3630070743 reap_dead start 2023-11-28T11:17:58.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f0d5bfff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5c0cd560 con 0x7f0d5c06c810 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f0d5affd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0d5c06c810 0x7f0d5c06ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46950/0 (socket says 172.21.15.53:46950) 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f0d5affd700 1 -- 172.21.15.53:0/2475786444 learned_addr learned my addr 172.21.15.53:0/2475786444 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f0d5a7fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d5c0cc020 0x7f0d5c1586e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43642/0 (socket says 172.21.15.53:43642) 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.739+0000 7f0d5b7fe700 1 --2- 172.21.15.53:0/2475786444 >> v2:172.21.15.53:3300/0 conn(0x7f0d5c0ccbe0 0x7f0d5c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.739+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5c0cacf0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.740+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c0cc790 con 0x7f0d5c0ccbe0 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.740+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350646446 0 0) 0x7f0d4c001110 con 0x7f0d5c06c810 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.740+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d500046c0 con 0x7f0d5c06c810 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.740+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122293024 0 0) 0x7f0d5c0cacf0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.740+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d4c001110 con 0x7f0d5c0cc020 2023-11-28T11:17:58.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.738+0000 7f67d27ff700 1 -- 172.21.15.53:0/2729702599 wait complete. 2023-11-28T11:17:58.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.741+0000 7f67d27ff700 1 Processor -- start 2023-11-28T11:17:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.741+0000 7f0d5b7fe700 1 -- 172.21.15.53:0/2475786444 >> v1:172.21.15.157:6789/0 conn(0x7f0d5c0cc020 legacy=0x7f0d5c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.741+0000 7f1d0259c700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 0x7f1cec004fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4018fd0 tx=0x7f1cec005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a4b0d8301c513a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.741+0000 7f0d5b7fe700 1 -- 172.21.15.53:0/2475786444 >> v1:172.21.15.53:6789/0 conn(0x7f0d5c06c810 legacy=0x7f0d5c06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.741+0000 7f0d5b7fe700 1 -- 172.21.15.53:0/2475786444 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c0cafd0 con 0x7f0d5c0ccbe0 2023-11-28T11:17:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.742+0000 7f0d5b7fe700 1 --2- 172.21.15.53:0/2475786444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0cd020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009d90 tx=0x7f0d5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e516a710d25315 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.742+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf401dca0 con 0x7f1cec004bf0 2023-11-28T11:17:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.742+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf401de20 con 0x7f1cec004bf0 2023-11-28T11:17:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.743+0000 7f67d27ff700 1 -- start start 2023-11-28T11:17:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.743+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400e070 con 0x7f0d5c0ccbe0 2023-11-28T11:17:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.743+0000 7f0d59ffb700 1 -- 172.21.15.53:0/2475786444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5400bb20 con 0x7f0d5c0ccbe0 2023-11-28T11:17:58.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.743+0000 7f1d0159a700 1 -- 172.21.15.53:0/3630070743 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4023660 con 0x7f1cec004bf0 2023-11-28T11:17:58.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.744+0000 7f1d08bff700 1 -- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 msgr2=0x7f1cec004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.744+0000 7f1d08bff700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 0x7f1cec004fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4018fd0 tx=0x7f1cec005880 comp rx=0 tx=0).stop 2023-11-28T11:17:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.744+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0ccbe0 msgr2=0x7f0d5c0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.744+0000 7f0d5bfff700 1 --2- 172.21.15.53:0/2475786444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0cd020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009d90 tx=0x7f0d5400b330 comp rx=0 tx=0).stop 2023-11-28T11:17:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.744+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 shutdown_connections 2023-11-28T11:17:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f0d5bfff700 1 --2- 172.21.15.53:0/2475786444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0cd020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 >> 172.21.15.53:0/2475786444 conn(0x7f0d5c0cb510 msgr2=0x7f0d5c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67d27ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67d27ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 0x7f67cc15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67d27ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc160f00 0x7f67cc163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f1d08bff700 1 -- 172.21.15.53:0/3630070743 shutdown_connections 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f1d08bff700 1 --2- 172.21.15.53:0/3630070743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cec004bf0 0x7f1cec004fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f1d08bff700 1 -- 172.21.15.53:0/3630070743 >> 172.21.15.53:0/3630070743 conn(0x7f1d0406ae80 msgr2=0x7f1d040b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67cb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 0x7f67cc15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67cb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 0x7f67cc15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38190/0 (socket says 172.21.15.53:38190) 2023-11-28T11:17:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67cb7fe700 1 -- 172.21.15.53:0/3274434391 learned_addr learned my addr 172.21.15.53:0/3274434391 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 shutdown_connections 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc162330 con 0x7f67cc160f00 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc162c40 con 0x7f67cc160b30 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc162f20 con 0x7f67cc0cc730 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f67d0d9c700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc160f00 0x7f67cc163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.745+0000 7f67cbfff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f1d08bff700 1 -- 172.21.15.53:0/3630070743 shutdown_connections 2023-11-28T11:17:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f1d08bff700 1 -- 172.21.15.53:0/3630070743 wait complete. 2023-11-28T11:17:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.747+0000 7f67cbfff700 1 -- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 msgr2=0x7f67cc15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.747+0000 7f67cbfff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 0x7f67cc15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.747+0000 7f67cbfff700 1 -- 172.21.15.53:0/3274434391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc160f00 msgr2=0x7f67cc163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.747+0000 7f67cbfff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc160f00 0x7f67cc163ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.747+0000 7f67cbfff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc009550 con 0x7f67cc0cc730 2023-11-28T11:17:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.748+0000 7f67cbfff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0c65b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67c400cea0 tx=0x7f67c400cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.746+0000 7f0d5bfff700 1 -- 172.21.15.53:0/2475786444 wait complete. 2023-11-28T11:17:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.749+0000 7f1d08bff700 1 Processor -- start 2023-11-28T11:17:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.749+0000 7f1d08bff700 1 -- start start 2023-11-28T11:17:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.750+0000 7f0d5bfff700 1 Processor -- start 2023-11-28T11:17:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f0d5bfff700 1 -- start start 2023-11-28T11:17:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ae50 con 0x7f67cc0cc730 2023-11-28T11:17:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4015440 con 0x7f67cc0cc730 2023-11-28T11:17:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40157d0 con 0x7f67cc0cc730 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d040c4160 0x7f1d04152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d040b3240 0x7f1d041539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b2570 con 0x7f1d040c4160 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b27a0 con 0x7f1d040b3240 2023-11-28T11:17:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d08bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040c4990 con 0x7f1cec004bf0 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60478/0 (socket says 172.21.15.53:60478) 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d0259c700 1 -- 172.21.15.53:0/788860851 learned_addr learned my addr 172.21.15.53:0/788860851 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.751+0000 7f1d02d9d700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d040b3240 0x7f1d041539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc0ccbb0 con 0x7f67cc0cc730 2023-11-28T11:17:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 -- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d040b3240 msgr2=0x7f1d041539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d040b3240 0x7f1d041539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 -- 172.21.15.53:0/788860851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d040c4160 msgr2=0x7f1d04152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d040c4160 0x7f1d04152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 -- 172.21.15.53:0/788860851 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040ce2d0 con 0x7f1cec004bf0 2023-11-28T11:17:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc165550 con 0x7f67cc0cc730 2023-11-28T11:17:58.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.752+0000 7f1d0259c700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4001660 tx=0x7f1cf4001930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.753+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 0x7f0d5c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.753+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.753+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 0x7f0d5c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.753+0000 7f0d5affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 0x7f0d5c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.753+0000 7f0d5bfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0b4840 con 0x7f0d5c0cc020 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5bfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1e7c60 con 0x7f0d5c0ccbe0 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5bfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0b2290 con 0x7f0d5c0b3330 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38204/0 (socket says 172.21.15.53:38204) 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5a7fc700 1 -- 172.21.15.53:0/3039716579 learned_addr learned my addr 172.21.15.53:0/3039716579 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40054e0 con 0x7f1cec004bf0 2023-11-28T11:17:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4017870 con 0x7f1cec004bf0 2023-11-28T11:17:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.754+0000 7f0d5b7fe700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 0x7f0d5c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4017c60 con 0x7f1cec004bf0 2023-11-28T11:17:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d040cc8e0 con 0x7f1cec004bf0 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f0d5affd700 1 -- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 msgr2=0x7f0d5c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f0d5affd700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 0x7f0d5c0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f0d5affd700 1 -- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 msgr2=0x7f0d5c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f0d5affd700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f0d5affd700 1 -- 172.21.15.53:0/3039716579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009a70 con 0x7f0d5c0cc020 2023-11-28T11:17:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.755+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040ce5b0 con 0x7f1cec004bf0 2023-11-28T11:17:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.756+0000 7f0d5b7fe700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 0x7f0d5c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.757+0000 7f0d5affd700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 0x7f0d5c0b1c00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c700 tx=0x7f0d4c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.756+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67c401c030 con 0x7f67cc0cc730 2023-11-28T11:17:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.757+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc0ccd40 con 0x7f67cc0cc730 2023-11-28T11:17:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.757+0000 7f67c8ff9700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 0x7f67b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d04061e80 con 0x7f1cec004bf0 2023-11-28T11:17:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c005e30 con 0x7f0d5c0cc020 2023-11-28T11:17:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67c405e370 con 0x7f67cc0cc730 2023-11-28T11:17:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f67cb7fe700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 0x7f67b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.759+0000 7f67cb7fe700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 0x7f67b806a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0c52c0 tx=0x7f67bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f0d5bfff700 1 -- 172.21.15.53:0/3039716579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c0ca6c0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f0d5bfff700 1 -- 172.21.15.53:0/3039716579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c0cb2b0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c0070d0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.758+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c007480 con 0x7f0d5c0cc020 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.760+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1cf4021050 con 0x7f1cec004bf0 2023-11-28T11:17:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.761+0000 7f1cf2ffd700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 0x7f1ce406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.762+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cf4098170 con 0x7f1cec004bf0 2023-11-28T11:17:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.762+0000 7f1d01d9b700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 0x7f1ce406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.762+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340052f0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.763+0000 7f1d01d9b700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 0x7f1ce406a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80010e0 tx=0x7f1cf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.764+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d4c019030 con 0x7f0d5c0cc020 2023-11-28T11:17:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.764+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f67c408b060 con 0x7f67cc0cc730 2023-11-28T11:17:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.765+0000 7f0d4b7fe700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 0x7f0d3c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.765+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d4c084fa0 con 0x7f0d5c0cc020 2023-11-28T11:17:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.766+0000 7f0d5a7fc700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 0x7f0d3c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.766+0000 7f0d5a7fc700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 0x7f0d3c06a7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d440011e0 tx=0x7f0d44007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.768+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1cf40648e0 con 0x7f1cec004bf0 2023-11-28T11:17:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.777+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d4c051790 con 0x7f0d5c0cc020 2023-11-28T11:17:58.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.835+0000 7f342f39d700 1 Processor -- start 2023-11-28T11:17:58.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.835+0000 7f342f39d700 1 -- start start 2023-11-28T11:17:58.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.837+0000 7f6cba7ff700 1 Processor -- start 2023-11-28T11:17:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.837+0000 7f342f39d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f342806c810 0x7f342815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.838+0000 7f342f39d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.838+0000 7f342f39d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34280caa10 con 0x7f3428158950 2023-11-28T11:17:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.838+0000 7f342eb9c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.838+0000 7f342eb9c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60494/0 (socket says 172.21.15.53:60494) 2023-11-28T11:17:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.838+0000 7f342eb9c700 1 -- 172.21.15.53:0/1986021585 learned_addr learned my addr 172.21.15.53:0/1986021585 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.837+0000 7f6cba7ff700 1 -- start start 2023-11-28T11:17:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34280cc790 con 0x7f342806c810 2023-11-28T11:17:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3428158d20 con 0x7f342815c8e0 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb40ce480 0x7f6cb40c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6cb4161220 0x7f6cb406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4164e60 0x7f6cb41652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb4162210 con 0x7f6cb4161220 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb41649d0 con 0x7f6cb40ce480 2023-11-28T11:17:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.839+0000 7f6cba7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb406c020 con 0x7f6cb4164e60 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342db9a700 1 --2- 172.21.15.53:0/1986021585 >> v2:172.21.15.157:3300/0 conn(0x7f342806c810 0x7f342815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342eb9c700 1 -- 172.21.15.53:0/1986021585 >> v2:172.21.15.157:3300/0 conn(0x7f342806c810 msgr2=0x7f342815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342eb9c700 1 --2- 172.21.15.53:0/1986021585 >> v2:172.21.15.157:3300/0 conn(0x7f342806c810 0x7f342815c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342eb9c700 1 -- 172.21.15.53:0/1986021585 >> v1:172.21.15.157:6789/0 conn(0x7f3428158950 legacy=0x7f342806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342eb9c700 1 -- 172.21.15.53:0/1986021585 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280cacf0 con 0x7f342815c8e0 2023-11-28T11:17:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f342eb9c700 1 --2- 172.21.15.53:0/1986021585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3418007360 tx=0x7f3418007670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9a004848ab999c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f6cb3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb40ce480 0x7f6cb40c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f6cb3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb40ce480 0x7f6cb40c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38228/0 (socket says 172.21.15.53:38228) 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f6cb3fff700 1 -- 172.21.15.53:0/1284440316 learned_addr learned my addr 172.21.15.53:0/1284440316 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f6cb8d9c700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4164e60 0x7f6cb41652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.840+0000 7f6cb37fe700 1 --2- 172.21.15.53:0/1284440316 >> v2:172.21.15.53:3300/0 conn(0x7f6cb4161220 0x7f6cb406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f342d399700 1 -- 172.21.15.53:0/1986021585 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800f070 con 0x7f342815c8e0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f342d399700 1 -- 172.21.15.53:0/1986021585 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418008bf0 con 0x7f342815c8e0 2023-11-28T11:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f342d399700 1 -- 172.21.15.53:0/1986021585 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418004030 con 0x7f342815c8e0 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f6cb3fff700 1 -- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4164e60 msgr2=0x7f6cb41652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f6cb3fff700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4164e60 0x7f6cb41652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f6cb3fff700 1 -- 172.21.15.53:0/1284440316 >> v2:172.21.15.53:3300/0 conn(0x7f6cb4161220 msgr2=0x7f6cb406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f6cb3fff700 1 --2- 172.21.15.53:0/1284440316 >> v2:172.21.15.53:3300/0 conn(0x7f6cb4161220 0x7f6cb406b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.841+0000 7f6cb3fff700 1 -- 172.21.15.53:0/1284440316 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb406bd60 con 0x7f6cb40ce480 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.842+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 msgr2=0x7f34280ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.842+0000 7f342f39d700 1 --2- 172.21.15.53:0/1986021585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3418007360 tx=0x7f3418007670 comp rx=0 tx=0).stop 2023-11-28T11:17:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.842+0000 7f9bf7fff700 1 Processor -- start 2023-11-28T11:17:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.842+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 shutdown_connections 2023-11-28T11:17:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.843+0000 7f9bf7fff700 1 -- start start 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f342f39d700 1 --2- 172.21.15.53:0/1986021585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f342815c8e0 0x7f34280ccbe0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f342f39d700 1 --2- 172.21.15.53:0/1986021585 >> v2:172.21.15.157:3300/0 conn(0x7f342806c810 0x7f342815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 >> 172.21.15.53:0/1986021585 conn(0x7f34280cb510 msgr2=0x7f34280cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 shutdown_connections 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f342f39d700 1 -- 172.21.15.53:0/1986021585 wait complete. 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.844+0000 7f6cb3fff700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb40ce480 0x7f6cb40c1fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40cc6a0 tx=0x7f6ca4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b023a6bf2c2ada server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.845+0000 7f6cb2ffd700 1 -- 172.21.15.53:0/1284440316 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011830 con 0x7f6cb40ce480 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.845+0000 7f6cb2ffd700 1 -- 172.21.15.53:0/1284440316 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca4011990 con 0x7f6cb40ce480 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.845+0000 7f6cb2ffd700 1 -- 172.21.15.53:0/1284440316 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011d20 con 0x7f6cb40ce480 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 0x7f9bf8071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf7fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf815e710 con 0x7f9bf80adf60 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf7fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf815ef30 con 0x7f9bf815ab80 2023-11-28T11:17:58.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf7fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815a890 con 0x7f9bf8071a60 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f342f39d700 1 Processor -- start 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf77fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf77fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60508/0 (socket says 172.21.15.53:60508) 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.846+0000 7f9bf77fe700 1 -- 172.21.15.53:0/383902861 learned_addr learned my addr 172.21.15.53:0/383902861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f9bf67fc700 1 --2- 172.21.15.53:0/383902861 >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 0x7f9bf8071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f9bf77fe700 1 -- 172.21.15.53:0/383902861 >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 msgr2=0x7f9bf8071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f9bf5ffb700 1 -- 172.21.15.53:0/383902861 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480227661 0 0) 0x7f9bf815e710 con 0x7f9bf80adf60 2023-11-28T11:17:58.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f6cba7ff700 1 -- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb40ce480 msgr2=0x7f6cb40c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb40ce480 0x7f6cb40c1fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40cc6a0 tx=0x7f6ca4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f9bf77fe700 1 --2- 172.21.15.53:0/383902861 >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 0x7f9bf8071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.847+0000 7f9bf77fe700 1 -- 172.21.15.53:0/383902861 >> v1:172.21.15.157:6789/0 conn(0x7f9bf80adf60 legacy=0x7f9bf815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf77fe700 1 -- 172.21.15.53:0/383902861 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf815e970 con 0x7f9bf8071a60 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf67fc700 1 --2- 172.21.15.53:0/383902861 >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 0x7f9bf8071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf77fe700 1 --2- 172.21.15.53:0/383902861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009e50 tx=0x7f9bf000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf222accbbca1781 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 -- 172.21.15.53:0/1284440316 shutdown_connections 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb4164e60 0x7f6cb41652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/1284440316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb40ce480 0x7f6cb40c1fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/1284440316 >> v2:172.21.15.53:3300/0 conn(0x7f6cb4161220 0x7f6cb406b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 -- 172.21.15.53:0/1284440316 >> 172.21.15.53:0/1284440316 conn(0x7f6cb40cb670 msgr2=0x7f6cb40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f342f39d700 1 -- start start 2023-11-28T11:17:58.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf5ffb700 1 -- 172.21.15.53:0/383902861 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000e070 con 0x7f9bf8071a60 2023-11-28T11:17:58.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f6cba7ff700 1 -- 172.21.15.53:0/1284440316 shutdown_connections 2023-11-28T11:17:58.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf5ffb700 1 -- 172.21.15.53:0/383902861 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf000b9c0 con 0x7f9bf8071a60 2023-11-28T11:17:58.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.848+0000 7f9bf5ffb700 1 -- 172.21.15.53:0/383902861 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000bd70 con 0x7f9bf8071a60 2023-11-28T11:17:58.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f6cba7ff700 1 -- 172.21.15.53:0/1284440316 wait complete. 2023-11-28T11:17:58.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f9bf7fff700 1 -- 172.21.15.53:0/383902861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 msgr2=0x7f9bf80b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f9bf7fff700 1 --2- 172.21.15.53:0/383902861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009e50 tx=0x7f9bf000b330 comp rx=0 tx=0).stop 2023-11-28T11:17:58.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f9bf7fff700 1 -- 172.21.15.53:0/383902861 shutdown_connections 2023-11-28T11:17:58.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f9bf7fff700 1 --2- 172.21.15.53:0/383902861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf8071a60 0x7f9bf80b0510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f9bf7fff700 1 --2- 172.21.15.53:0/383902861 >> v2:172.21.15.53:3300/0 conn(0x7f9bf815ab80 0x7f9bf8071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f9bf7fff700 1 -- 172.21.15.53:0/383902861 >> 172.21.15.53:0/383902861 conn(0x7f9bf80cb510 msgr2=0x7f9bf80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342815c8e0 0x7f34280b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3428155190 0x7f34280b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b4680 con 0x7f3428155190 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b40c0 con 0x7f342815c8e0 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.849+0000 7f342f39d700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b43a0 con 0x7f3428158950 2023-11-28T11:17:58.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60512/0 (socket says 172.21.15.53:60512) 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 -- 172.21.15.53:0/614471829 learned_addr learned my addr 172.21.15.53:0/614471829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 -- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342815c8e0 msgr2=0x7f34280b2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342815c8e0 0x7f34280b2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 -- 172.21.15.53:0/614471829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3428155190 msgr2=0x7f34280b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3428155190 0x7f34280b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.850+0000 7f342e39b700 1 -- 172.21.15.53:0/614471829 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418007040 con 0x7f3428158950 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f342e39b700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f342400c4d0 tx=0x7f342400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424011790 con 0x7f3428158950 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34240118f0 con 0x7f3428158950 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424011ca0 con 0x7f3428158950 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34280cb2b0 con 0x7f3428158950 2023-11-28T11:17:58.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.851+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34280cafd0 con 0x7f3428158950 2023-11-28T11:17:58.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.852+0000 7f6cba7ff700 1 Processor -- start 2023-11-28T11:17:58.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.853+0000 7f9bf7fff700 1 -- 172.21.15.53:0/383902861 shutdown_connections 2023-11-28T11:17:58.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.853+0000 7f9bf7fff700 1 -- 172.21.15.53:0/383902861 wait complete. 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.853+0000 7f6cba7ff700 1 -- start start 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb40ce480 0x7f6cb4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb4164e60 0x7f6cb40b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb41558c0 con 0x7f6cb4164e60 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c8be0 con 0x7f6cb4161220 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f6cba7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40b3b50 con 0x7f6cb40ce480 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.855+0000 7f6cb37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.855+0000 7f6cb37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38242/0 (socket says 172.21.15.53:38242) 2023-11-28T11:17:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.855+0000 7f6cb37fe700 1 -- 172.21.15.53:0/3268970564 learned_addr learned my addr 172.21.15.53:0/3268970564 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.855+0000 7f6cb8d9c700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb4164e60 0x7f6cb40b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.854+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3424010050 con 0x7f3428158950 2023-11-28T11:17:58.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.855+0000 7f9bf7fff700 1 Processor -- start 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 -- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb40ce480 msgr2=0x7f6cb4155380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb40ce480 0x7f6cb4155380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 -- 172.21.15.53:0/3268970564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb4164e60 msgr2=0x7f6cb40b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb4164e60 0x7f6cb40b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 -- 172.21.15.53:0/3268970564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009550 con 0x7f6cb4161220 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb37fe700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e870 tx=0x7f6ca800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8009590 con 0x7f6cb4161220 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.856+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8013680 con 0x7f6cb4161220 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb4161970 con 0x7f6cb4161220 2023-11-28T11:17:58.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80139f0 con 0x7f6cb4161220 2023-11-28T11:17:58.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34280ca6c0 con 0x7f3428158950 2023-11-28T11:17:58.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb41616a0 con 0x7f6cb4161220 2023-11-28T11:17:58.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f341effd700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 0x7f340c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.857+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3424085a70 con 0x7f3428158950 2023-11-28T11:17:58.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.858+0000 7f342db9a700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 0x7f340c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.858+0000 7f9bf7fff700 1 -- start start 2023-11-28T11:17:58.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.859+0000 7f342db9a700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 0x7f340c06a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34281569a0 tx=0x7f3420007310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.859+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 0x7f9bf81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.859+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 0x7f9bf80c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.859+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 0x7f9bf80c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf7fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c34f0 con 0x7f9bf8071a60 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf7fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c2f30 con 0x7f9bf815ab80 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf7fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c3210 con 0x7f9bf80adf60 2023-11-28T11:17:58.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4161c50 con 0x7f6cb4161220 2023-11-28T11:17:58.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf67fc700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 0x7f9bf80c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 0x7f9bf80c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf67fc700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 0x7f9bf80c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60528/0 (socket says 172.21.15.53:60528) 2023-11-28T11:17:58.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf67fc700 1 -- 172.21.15.53:0/1921338296 learned_addr learned my addr 172.21.15.53:0/1921338296 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf6ffd700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 0x7f9bf81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf77fe700 1 -- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 msgr2=0x7f9bf80c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf77fe700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 0x7f9bf80c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.860+0000 7f9bf77fe700 1 -- 172.21.15.53:0/1921338296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 msgr2=0x7f9bf81502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.861+0000 7f9bf77fe700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 0x7f9bf81502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.861+0000 7f9bf77fe700 1 -- 172.21.15.53:0/1921338296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009b30 con 0x7f9bf815ab80 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.861+0000 7f9bf6ffd700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 0x7f9bf81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:17:58.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.861+0000 7f9bf77fe700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 0x7f9bf80c4910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0015000 tx=0x7f9bf0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.861+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ca8023070 con 0x7f6cb4161220 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.862+0000 7f6cb0ff9700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 0x7f6ca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.862+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ca8085e50 con 0x7f6cb4161220 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.863+0000 7f6cb3fff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 0x7f6ca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.864+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34240521e0 con 0x7f3428158950 2023-11-28T11:17:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.864+0000 7f6cb3fff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 0x7f6ca006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4014040 tx=0x7f6ca4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.866+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003ea0 con 0x7f9bf815ab80 2023-11-28T11:17:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.867+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0004930 con 0x7f9bf815ab80 2023-11-28T11:17:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.867+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf001c680 con 0x7f9bf815ab80 2023-11-28T11:17:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.867+0000 7f9bf7fff700 1 -- 172.21.15.53:0/1921338296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf8070d20 con 0x7f9bf815ab80 2023-11-28T11:17:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.867+0000 7f9bf7fff700 1 -- 172.21.15.53:0/1921338296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf815ec50 con 0x7f9bf815ab80 2023-11-28T11:17:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.867+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ca8052560 con 0x7f6cb4161220 2023-11-28T11:17:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.871+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9bf0022030 con 0x7f9bf815ab80 2023-11-28T11:17:58.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.872+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8071000 con 0x7f9bf815ab80 2023-11-28T11:17:58.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.872+0000 7f9bef7fe700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 0x7f9bd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.873+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9bf005fba0 con 0x7f9bf815ab80 2023-11-28T11:17:58.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.873+0000 7f9bf6ffd700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 0x7f9bd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.874+0000 7f9bf6ffd700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 0x7f9bd806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9be8000f60 tx=0x7f9be8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.879+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9bf00545e0 con 0x7f9bf815ab80 2023-11-28T11:17:58.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.886+0000 7f19625ff700 1 Processor -- start 2023-11-28T11:17:58.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.887+0000 7f19625ff700 1 -- start start 2023-11-28T11:17:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.889+0000 7f19625ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f195c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.890+0000 7f195b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f195c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.890+0000 7f1960b9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f195c0ce2e0 0x7f195c06c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46964/0 (socket says 172.21.15.53:46964) 2023-11-28T11:17:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.890+0000 7f1960b9c700 1 -- 172.21.15.53:0/966844708 learned_addr learned my addr 172.21.15.53:0/966844708 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.890+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c06bf10 con 0x7f195c0ce2e0 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.891+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c0ce720 con 0x7f195c158ce0 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.891+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1590b0 con 0x7f195c06fb30 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.891+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666376647 0 0) 0x7f195c06bf10 con 0x7f195c0ce2e0 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.891+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19440057c0 con 0x7f195c0ce2e0 2023-11-28T11:17:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.891+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967059171 0 0) 0x7f195c0ce720 con 0x7f195c158ce0 2023-11-28T11:17:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f195c06bf10 con 0x7f195c158ce0 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1701388114 0 0) 0x7f19440057c0 con 0x7f195c0ce2e0 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 msgr2=0x7f195c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f195c0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 >> v1:172.21.15.157:6789/0 conn(0x7f195c158ce0 legacy=0x7f195c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0cb0d0 con 0x7f195c0ce2e0 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f19540019f0 con 0x7f195c0ce2e0 2023-11-28T11:17:58.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 >> v1:172.21.15.53:6789/0 conn(0x7f195c0ce2e0 legacy=0x7f195c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1944006e30 0x7f1944007260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f1944005e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f19440063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1590b0 con 0x7f1944006e30 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194400bc10 con 0x7f1944005a00 2023-11-28T11:17:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.892+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194400bdd0 con 0x7f195c06fb30 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.893+0000 7f1960b9c700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f19440063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195b7fe700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f1944005e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195bfff700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1944006e30 0x7f1944007260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195b7fe700 1 -- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 msgr2=0x7f19440063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195b7fe700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c06fb30 0x7f19440063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195b7fe700 1 -- 172.21.15.53:0/966844708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1944006e30 msgr2=0x7f1944007260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.894+0000 7f195b7fe700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1944006e30 0x7f1944007260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.895+0000 7f195b7fe700 1 -- 172.21.15.53:0/966844708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0ce720 con 0x7f1944005a00 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.895+0000 7f195b7fe700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f1944005e70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f194c009b60 tx=0x7f1944007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3683f453486a667 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.895+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00e6c0 con 0x7f1944005a00 2023-11-28T11:17:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.895+0000 7f195affd700 1 -- 172.21.15.53:0/966844708 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00e820 con 0x7f1944005a00 2023-11-28T11:17:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.896+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 msgr2=0x7f1944005e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.896+0000 7f19625ff700 1 --2- 172.21.15.53:0/966844708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f1944005e70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f194c009b60 tx=0x7f1944007a90 comp rx=0 tx=0).stop 2023-11-28T11:17:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.897+0000 7f1b111ff700 1 Processor -- start 2023-11-28T11:17:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.896+0000 7f1960b9c700 1 -- 172.21.15.53:0/966844708 reap_dead start 2023-11-28T11:17:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.897+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 shutdown_connections 2023-11-28T11:17:58.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.897+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 >> 172.21.15.53:0/966844708 conn(0x7f195c0cb670 msgr2=0x7f195c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.897+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 shutdown_connections 2023-11-28T11:17:58.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.898+0000 7f19625ff700 1 -- 172.21.15.53:0/966844708 wait complete. 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.897+0000 7f1b111ff700 1 -- start start 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.899+0000 7f1b111ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b0c0d1330 0x7f1b0c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b111ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b111ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0b0230 con 0x7f1b0c0d01b0 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b111ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c15cec0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b0b59e700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b0b59e700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:60554/0 (socket says 172.21.15.53:60554) 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.900+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1043034643 learned_addr learned my addr 172.21.15.53:0/1043034643 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.901+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0b4570 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.901+0000 7f19625ff700 1 Processor -- start 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.901+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1043034643 >> v2:172.21.15.53:3300/0 conn(0x7f1b0c0d1330 msgr2=0x7f1b0c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.902+0000 7f1b0b59e700 1 --2- 172.21.15.53:0/1043034643 >> v2:172.21.15.53:3300/0 conn(0x7f1b0c0d1330 0x7f1b0c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.902+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1043034643 >> v1:172.21.15.53:6789/0 conn(0x7f1b0c0d01b0 legacy=0x7f1b0c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.902+0000 7f1b09d9b700 1 -- 172.21.15.53:0/1043034643 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818833870 0 0) 0x7f1b0c0b0230 con 0x7f1b0c0d01b0 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.902+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1043034643 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0b4850 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.903+0000 7f19625ff700 1 -- start start 2023-11-28T11:17:58.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.903+0000 7f1b0b59e700 1 --2- 172.21.15.53:0/1043034643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009e50 tx=0x7f1b0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf484be149bd2515 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.904+0000 7f1b09d9b700 1 -- 172.21.15.53:0/1043034643 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400e070 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.904+0000 7f1b09d9b700 1 -- 172.21.15.53:0/1043034643 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0400b9c0 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.904+0000 7f1b09d9b700 1 -- 172.21.15.53:0/1043034643 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400bd70 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.905+0000 7f19625ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f195c1ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.905+0000 7f19625ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0c6440 0x7f195c1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f19625ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f19625ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0b1a20 con 0x7f195c1f5d90 2023-11-28T11:17:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f19625ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1ff960 con 0x7f1944005a00 2023-11-28T11:17:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f19625ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1ffc40 con 0x7f195c0c6440 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 msgr2=0x7f1b0c0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1043034643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009e50 tx=0x7f1b0400b330 comp rx=0 tx=0).stop 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 shutdown_connections 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1043034643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0b0c00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1043034643 >> v2:172.21.15.53:3300/0 conn(0x7f1b0c0d1330 0x7f1b0c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.906+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 >> 172.21.15.53:0/1043034643 conn(0x7f1b0c09c3f0 msgr2=0x7f1b0c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 shutdown_connections 2023-11-28T11:17:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f1960b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f1960b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40258/0 (socket says 172.21.15.53:40258) 2023-11-28T11:17:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f1960b9c700 1 -- 172.21.15.53:0/1791187887 learned_addr learned my addr 172.21.15.53:0/1791187887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f195b7fe700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0c6440 0x7f195c1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.908+0000 7f1960b9c700 1 -- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0c6440 msgr2=0x7f195c1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.908+0000 7f1960b9c700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0c6440 0x7f195c1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.908+0000 7f1960b9c700 1 -- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 msgr2=0x7f195c1ff110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:17:58.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.908+0000 7f1960b9c700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f195c1ff110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.908+0000 7f1960b9c700 1 -- 172.21.15.53:0/1791187887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c06b8c0 con 0x7f195c1f5d90 2023-11-28T11:17:58.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.909+0000 7f1960b9c700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f195c0cebc0 tx=0x7f1954007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.907+0000 7f1b111ff700 1 -- 172.21.15.53:0/1043034643 wait complete. 2023-11-28T11:17:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.910+0000 7f1b111ff700 1 Processor -- start 2023-11-28T11:17:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.910+0000 7f1b111ff700 1 -- start start 2023-11-28T11:17:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.910+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400acb0 con 0x7f195c1f5d90 2023-11-28T11:17:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.911+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c0cb3b0 con 0x7f195c1f5d90 2023-11-28T11:17:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.911+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c06bba0 con 0x7f195c1f5d90 2023-11-28T11:17:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.911+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195400ae10 con 0x7f195c1f5d90 2023-11-28T11:17:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.911+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954013630 con 0x7f195c1f5d90 2023-11-28T11:17:58.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.913+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c0052f0 con 0x7f195c1f5d90 2023-11-28T11:17:58.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 0x7f1b0c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c1feb90 con 0x7f1b0c0d1330 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b0a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cc6a0 con 0x7f1b0c0d01b0 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b111ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cc980 con 0x7f1b0c0b20b0 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b0b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 0x7f1b0c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b0a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38276/0 (socket says 172.21.15.53:38276) 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.914+0000 7f1b0a59c700 1 -- 172.21.15.53:0/1033783675 learned_addr learned my addr 172.21.15.53:0/1033783675 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0ad9d700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 msgr2=0x7f1b0c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0b59e700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 msgr2=0x7f1b0c06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:58.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0b59e700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0b59e700 1 -- 172.21.15.53:0/1033783675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009b30 con 0x7f1b0c0d1330 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.915+0000 7f1b0a59c700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.916+0000 7f1b0b59e700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 0x7f1b0c06b7e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b04015000 tx=0x7f1b04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.916+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004930 con 0x7f1b0c0d1330 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.916+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c0afec0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.917+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0afbe0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.917+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b040099b0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.919+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec0052f0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.919+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1954013790 con 0x7f195c1f5d90 2023-11-28T11:17:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.921+0000 7f1958ff9700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 0x7f194806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.921+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1954086a80 con 0x7f195c1f5d90 2023-11-28T11:17:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.922+0000 7f195bfff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 0x7f194806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.922+0000 7f195bfff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 0x7f194806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1950007fd0 tx=0x7f1950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.923+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004930 con 0x7f1b0c0d1330 2023-11-28T11:17:58.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.923+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b04022030 con 0x7f1b0c0d1330 2023-11-28T11:17:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.924+0000 7f1af37fe700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 0x7f1af806a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.925+0000 7f1b0ad9d700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 0x7f1af806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.925+0000 7f1b0ad9d700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 0x7f1af806a800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000f60 tx=0x7f1afc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.931+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b0405fdf0 con 0x7f1b0c0d1330 2023-11-28T11:17:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.931+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1954056c20 con 0x7f195c1f5d90 2023-11-28T11:17:58.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:58.940+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b04054040 con 0x7f1b0c0d1330 2023-11-28T11:17:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.227+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1d040b1110 con 0x7f1cec004bf0 2023-11-28T11:17:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.227+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0d340055a0 con 0x7f0d5c0cc020 2023-11-28T11:17:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.228+0000 7f1cf2ffd700 1 -- 172.21.15.53:0/788860851 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1cf4067f00 con 0x7f1cec004bf0 2023-11-28T11:17:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.228+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f67cc1630b0 con 0x7f67cc0cc730 2023-11-28T11:17:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.229+0000 7f0d4b7fe700 1 -- 172.21.15.53:0/3039716579 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d4c054db0 con 0x7f0d5c0cc020 2023-11-28T11:17:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.231+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 msgr2=0x7f1ce406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.231+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 0x7f1ce406a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80010e0 tx=0x7f1cf8009040 comp rx=0 tx=0).stop 2023-11-28T11:17:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.231+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 msgr2=0x7f1d040cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.231+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4001660 tx=0x7f1cf4001930 comp rx=0 tx=0).stop 2023-11-28T11:17:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f67c8ff9700 1 -- 172.21.15.53:0/3274434391 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67c401a090 con 0x7f67cc0cc730 2023-11-28T11:17:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 shutdown_connections 2023-11-28T11:17:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce40683c0 0x7f1ce406a870 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80010e0 tx=0x7f1cf8009040 comp rx=0 tx=0).stop 2023-11-28T11:17:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cec004bf0 0x7f1d040cf380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d040b3240 0x7f1d041539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 --2- 172.21.15.53:0/788860851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d040c4160 0x7f1d04152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.232+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 >> 172.21.15.53:0/788860851 conn(0x7f1d0406ae80 msgr2=0x7f1d040af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.233+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 shutdown_connections 2023-11-28T11:17:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.233+0000 7f1d08bff700 1 -- 172.21.15.53:0/788860851 wait complete. 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 msgr2=0x7f0d3c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 0x7f0d3c06a7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d440011e0 tx=0x7f0d44007e80 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 msgr2=0x7f0d5c0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 0x7f0d5c0b1c00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c700 tx=0x7f0d4c00ca10 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 shutdown_connections 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d3c068340 0x7f0d3c06a7f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d5c0b3330 0x7f0d5c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c0cc020 0x7f0d5c0b1c00 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c700 tx=0x7f0d4c00ca10 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 --2- 172.21.15.53:0/3039716579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d5c0ccbe0 0x7f0d5c0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 >> 172.21.15.53:0/3039716579 conn(0x7f0d5c0cb510 msgr2=0x7f0d5c159700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.234+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 shutdown_connections 2023-11-28T11:17:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.235+0000 7f0d497fa700 1 -- 172.21.15.53:0/3039716579 wait complete. 2023-11-28T11:17:59.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.235+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 msgr2=0x7f67b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.235+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 0x7f67b806a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0c52c0 tx=0x7f67bc008030 comp rx=0 tx=0).stop 2023-11-28T11:17:59.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.235+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 msgr2=0x7f67cc0c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.235+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0c65b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67c400cea0 tx=0x7f67c400cf80 comp rx=0 tx=0).stop 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 shutdown_connections 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67b80682f0 0x7f67b806a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67cc0cc730 0x7f67cc0c65b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc160f00 0x7f67cc163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 --2- 172.21.15.53:0/3274434391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67cc160b30 0x7f67cc15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.237+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 >> 172.21.15.53:0/3274434391 conn(0x7f67cc0cb670 msgr2=0x7f67cc160120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.238+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 shutdown_connections 2023-11-28T11:17:59.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.238+0000 7f67d27ff700 1 -- 172.21.15.53:0/3274434391 wait complete. 2023-11-28T11:17:59.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2023-11-28T11:17:59.270 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-11-28T11:17:59.270 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2023-11-28T11:17:59.274 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.320+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f193c0055a0 con 0x7f195c1f5d90 2023-11-28T11:17:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.321+0000 7f1958ff9700 1 -- 172.21.15.53:0/1791187887 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1954056810 con 0x7f195c1f5d90 2023-11-28T11:17:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:60129542149 2023-11-28T11:17:59.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.321+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6cb40b3e30 con 0x7f6cb4161220 2023-11-28T11:17:59.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.322+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9bf8061e80 con 0x7f9bf815ab80 2023-11-28T11:17:59.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.322+0000 7f6cb0ff9700 1 -- 172.21.15.53:0/3268970564 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ca8055b80 con 0x7f6cb4161220 2023-11-28T11:17:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:59.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.322+0000 7f9bef7fe700 1 -- 172.21.15.53:0/1921338296 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9bf0054780 con 0x7f9bf815ab80 2023-11-28T11:17:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 msgr2=0x7f194806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 0x7f194806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1950007fd0 tx=0x7f1950009040 comp rx=0 tx=0).stop 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 msgr2=0x7f195c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f195c0cebc0 tx=0x7f1954007c90 comp rx=0 tx=0).stop 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 shutdown_connections 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19480682f0 0x7f194806a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c1f5d90 0x7f195c154330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1944005a00 0x7f195c1ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 --2- 172.21.15.53:0/1791187887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0c6440 0x7f195c1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 >> 172.21.15.53:0/1791187887 conn(0x7f195c0cb670 msgr2=0x7f195c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 shutdown_connections 2023-11-28T11:17:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.324+0000 7f19625ff700 1 -- 172.21.15.53:0/1791187887 wait complete. 2023-11-28T11:17:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f34280b47f0 con 0x7f3428158950 2023-11-28T11:17:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 msgr2=0x7f9bd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 0x7f9bd806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9be8000f60 tx=0x7f9be8009040 comp rx=0 tx=0).stop 2023-11-28T11:17:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 msgr2=0x7f9bf80c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 0x7f9bf80c4910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0015000 tx=0x7f9bf0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 shutdown_connections 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9bd80682f0 0x7f9bd806a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bf815ab80 0x7f9bf80c4910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf8071a60 0x7f9bf81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 --2- 172.21.15.53:0/1921338296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bf80adf60 0x7f9bf80c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.325+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 >> 172.21.15.53:0/1921338296 conn(0x7f9bf80cb510 msgr2=0x7f9bf815d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 shutdown_connections 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f9bed7fa700 1 -- 172.21.15.53:0/1921338296 wait complete. 2023-11-28T11:17:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 msgr2=0x7f6ca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 0x7f6ca006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4014040 tx=0x7f6ca4008030 comp rx=0 tx=0).stop 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 msgr2=0x7f6cb40c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e870 tx=0x7f6ca800eb80 comp rx=0 tx=0).stop 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 shutdown_connections 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6ca00682f0 0x7f6ca006a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cb40ce480 0x7f6cb4155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f341effd700 1 -- 172.21.15.53:0/614471829 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3424055800 con 0x7f3428158950 2023-11-28T11:17:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb4164e60 0x7f6cb40b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 --2- 172.21.15.53:0/3268970564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb4161220 0x7f6cb40c6050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 >> 172.21.15.53:0/3268970564 conn(0x7f6cb40cb670 msgr2=0x7f6cb40c1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.330 INFO:teuthology.orchestra.run.smithi053.stdout:60129542151 2023-11-28T11:17:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 shutdown_connections 2023-11-28T11:17:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.326+0000 7f6cba7ff700 1 -- 172.21.15.53:0/3268970564 wait complete. 2023-11-28T11:17:59.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.330+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 msgr2=0x7f340c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.330+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 0x7f340c06a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34281569a0 tx=0x7f3420007310 comp rx=0 tx=0).stop 2023-11-28T11:17:59.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.330+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 msgr2=0x7f34280c3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.330+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f342400c4d0 tx=0x7f342400c890 comp rx=0 tx=0).stop 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 shutdown_connections 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f340c0682f0 0x7f340c06a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3428158950 0x7f34280c3020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3428155190 0x7f34280b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 --2- 172.21.15.53:0/614471829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f342815c8e0 0x7f34280b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 >> 172.21.15.53:0/614471829 conn(0x7f34280cb510 msgr2=0x7f34280b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 shutdown_connections 2023-11-28T11:17:59.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.331+0000 7f342f39d700 1 -- 172.21.15.53:0/614471829 wait complete. 2023-11-28T11:17:59.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2023-11-28T11:17:59.349 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2023-11-28T11:17:59.351 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.352+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1aec005870 con 0x7f1b0c0d1330 2023-11-28T11:17:59.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.352+0000 7f1af37fe700 1 -- 172.21.15.53:0/1033783675 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b04057660 con 0x7f1b0c0d1330 2023-11-28T11:17:59.353 INFO:teuthology.orchestra.run.smithi053.stdout:60129542150 2023-11-28T11:17:59.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 msgr2=0x7f1af806a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 0x7f1af806a800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000f60 tx=0x7f1afc009040 comp rx=0 tx=0).stop 2023-11-28T11:17:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 msgr2=0x7f1b0c06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 0x7f1b0c06b7e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b04015000 tx=0x7f1b04004ad0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 shutdown_connections 2023-11-28T11:17:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1af8068350 0x7f1af806a800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0c0b20b0 0x7f1b0c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b0c0d1330 0x7f1b0c06b7e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 --2- 172.21.15.53:0/1033783675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0c0d01b0 0x7f1b0c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 >> 172.21.15.53:0/1033783675 conn(0x7f1b0c09c3f0 msgr2=0x7f1b0c15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 shutdown_connections 2023-11-28T11:17:59.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.354+0000 7f1b111ff700 1 -- 172.21.15.53:0/1033783675 wait complete. 2023-11-28T11:17:59.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2023-11-28T11:17:59.359 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2023-11-28T11:17:59.362 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-11-28T11:17:59.374 DEBUG:teuthology.parallel:result is None 2023-11-28T11:17:59.375 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:17:59.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:17:59.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.615+0000 7f22379ff700 1 Processor -- start 2023-11-28T11:17:59.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.615+0000 7f22379ff700 1 -- start start 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f22379ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f22300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f22379ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2230161340 0x7f22300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f22379ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300d1950 con 0x7f22300d2320 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f22379ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300b5800 con 0x7f2230161340 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f22379ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b5520 con 0x7f223015c9b0 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.616+0000 7f2234f9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2230161340 0x7f22300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2230161340 0x7f22300d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40276/0 (socket says 172.21.15.53:40276) 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 -- 172.21.15.53:0/478841560 learned_addr learned my addr 172.21.15.53:0/478841560 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f223579b700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f22300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 -- 172.21.15.53:0/478841560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 msgr2=0x7f22300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:17:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f22300b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 -- 172.21.15.53:0/478841560 >> v1:172.21.15.157:6789/0 conn(0x7f22300d2320 legacy=0x7f22300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 -- 172.21.15.53:0/478841560 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300d1300 con 0x7f2230161340 2023-11-28T11:17:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2227fff700 1 -- 172.21.15.53:0/478841560 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898106947 0 0) 0x7f22300d1950 con 0x7f22300d2320 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2234f9a700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22300d1de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22200098f0 tx=0x7f2220009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c388d225091906c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.617+0000 7f2227fff700 1 -- 172.21.15.53:0/478841560 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220011830 con 0x7f2230161340 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f2227fff700 1 -- 172.21.15.53:0/478841560 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2220011990 con 0x7f2230161340 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f2227fff700 1 -- 172.21.15.53:0/478841560 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220011d20 con 0x7f2230161340 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 -- 172.21.15.53:0/478841560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 msgr2=0x7f22300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22300d1de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22200098f0 tx=0x7f2220009c00 comp rx=0 tx=0).stop 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 -- 172.21.15.53:0/478841560 shutdown_connections 2023-11-28T11:17:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f22300b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 --2- 172.21.15.53:0/478841560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22300d1de0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 -- 172.21.15.53:0/478841560 >> 172.21.15.53:0/478841560 conn(0x7f2230072600 msgr2=0x7f22300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 -- 172.21.15.53:0/478841560 shutdown_connections 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.618+0000 7f22379ff700 1 -- 172.21.15.53:0/478841560 wait complete. 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.620+0000 7f22379ff700 1 Processor -- start 2023-11-28T11:17:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.620+0000 7f22379ff700 1 -- start start 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22300d2320 0x7f2230154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f2230150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2230150eb0 con 0x7f2230161340 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301f3230 con 0x7f22300d2320 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f22379ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2230155190 con 0x7f223015c9b0 2023-11-28T11:17:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f2235f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f2235f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40278/0 (socket says 172.21.15.53:40278) 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f2235f9c700 1 -- 172.21.15.53:0/1434774884 learned_addr learned my addr 172.21.15.53:0/1434774884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f2234f9a700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f2230150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.621+0000 7f223579b700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22300d2320 0x7f2230154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 -- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 msgr2=0x7f2230150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f2230150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 -- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22300d2320 msgr2=0x7f2230154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22300d2320 0x7f2230154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 -- 172.21.15.53:0/1434774884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220009550 con 0x7f2230161340 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2235f9c700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f222800c7a0 tx=0x7f222800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228003e30 con 0x7f2230161340 2023-11-28T11:17:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228005070 con 0x7f2230161340 2023-11-28T11:17:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228005420 con 0x7f2230161340 2023-11-28T11:17:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22300b5130 con 0x7f2230161340 2023-11-28T11:17:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.622+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22300d15e0 con 0x7f2230161340 2023-11-28T11:17:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.623+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22300b52c0 con 0x7f2230161340 2023-11-28T11:17:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.624+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2228019030 con 0x7f2230161340 2023-11-28T11:17:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.625+0000 7f2225ffb700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:17:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.625+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22280851e0 con 0x7f2230161340 2023-11-28T11:17:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.625+0000 7f223579b700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:17:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.626+0000 7f223579b700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 0x7f221406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f222c009b40 tx=0x7f222c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:17:59.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.629+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2228051950 con 0x7f2230161340 2023-11-28T11:17:59.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.829+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22300745c0 con 0x7f22140682f0 2023-11-28T11:17:59.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.838+0000 7f2225ffb700 1 -- 172.21.15.53:0/1434774884 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286613 (secure 0 0 0) 0x7f22300745c0 con 0x7f22140682f0 2023-11-28T11:17:59.842 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:17:59.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 msgr2=0x7f221406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 0x7f221406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f222c009b40 tx=0x7f222c009210 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 msgr2=0x7f22301f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f222800c7a0 tx=0x7f222800cab0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 shutdown_connections 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2230161340 0x7f22301f2cf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22300d2320 0x7f2230154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 --2- 172.21.15.53:0/1434774884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223015c9b0 0x7f2230150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 >> 172.21.15.53:0/1434774884 conn(0x7f2230072600 msgr2=0x7f223015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 shutdown_connections 2023-11-28T11:17:59.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:17:59.842+0000 7f22379ff700 1 -- 172.21.15.53:0/1434774884 wait complete. 2023-11-28T11:17:59.846 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:17:59.861 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T11:17:58.933572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":98912,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":95872,"kb_avail":754875808,"statfs":{"total":773094113280,"available":772992827392,"internally_reserved":0,"allocated":3047424,"data_stored":1845653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":98172928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.741265"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964019+0000","last_change":"2023-11-28T11:17:49.213974+0000","last_active":"2023-11-28T11:17:50.964019+0000","last_peered":"2023-11-28T11:17:50.964019+0000","last_clean":"2023-11-28T11:17:50.964019+0000","last_became_active":"2023-11-28T11:17:49.213506+0000","last_became_peered":"2023-11-28T11:17:49.213506+0000","last_unstale":"2023-11-28T11:17:50.964019+0000","last_undegraded":"2023-11-28T11:17:50.964019+0000","last_fullsized":"2023-11-28T11:17:50.964019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199622+0000","last_change":"2023-11-28T11:17:49.228867+0000","last_active":"2023-11-28T11:17:50.199622+0000","last_peered":"2023-11-28T11:17:50.199622+0000","last_clean":"2023-11-28T11:17:50.199622+0000","last_became_active":"2023-11-28T11:17:49.228581+0000","last_became_peered":"2023-11-28T11:17:49.228581+0000","last_unstale":"2023-11-28T11:17:50.199622+0000","last_undegraded":"2023-11-28T11:17:50.199622+0000","last_fullsized":"2023-11-28T11:17:50.199622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:10.291664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200594+0000","last_change":"2023-11-28T11:17:49.223726+0000","last_active":"2023-11-28T11:17:50.200594+0000","last_peered":"2023-11-28T11:17:50.200594+0000","last_clean":"2023-11-28T11:17:50.200594+0000","last_became_active":"2023-11-28T11:17:49.222194+0000","last_became_peered":"2023-11-28T11:17:49.222194+0000","last_unstale":"2023-11-28T11:17:50.200594+0000","last_undegraded":"2023-11-28T11:17:50.200594+0000","last_fullsized":"2023-11-28T11:17:50.200594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:23.103321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967010+0000","last_change":"2023-11-28T11:17:49.214435+0000","last_active":"2023-11-28T11:17:50.967010+0000","last_peered":"2023-11-28T11:17:50.967010+0000","last_clean":"2023-11-28T11:17:50.967010+0000","last_became_active":"2023-11-28T11:17:49.214093+0000","last_became_peered":"2023-11-28T11:17:49.214093+0000","last_unstale":"2023-11-28T11:17:50.967010+0000","last_undegraded":"2023-11-28T11:17:50.967010+0000","last_fullsized":"2023-11-28T11:17:50.967010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972744+0000","last_change":"2023-11-28T11:17:49.217415+0000","last_active":"2023-11-28T11:17:50.972744+0000","last_peered":"2023-11-28T11:17:50.972744+0000","last_clean":"2023-11-28T11:17:50.972744+0000","last_became_active":"2023-11-28T11:17:49.217134+0000","last_became_peered":"2023-11-28T11:17:49.217134+0000","last_unstale":"2023-11-28T11:17:50.972744+0000","last_undegraded":"2023-11-28T11:17:50.972744+0000","last_fullsized":"2023-11-28T11:17:50.972744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200106+0000","last_change":"2023-11-28T11:17:49.208771+0000","last_active":"2023-11-28T11:17:50.200106+0000","last_peered":"2023-11-28T11:17:50.200106+0000","last_clean":"2023-11-28T11:17:50.200106+0000","last_became_active":"2023-11-28T11:17:49.208484+0000","last_became_peered":"2023-11-28T11:17:49.208484+0000","last_unstale":"2023-11-28T11:17:50.200106+0000","last_undegraded":"2023-11-28T11:17:50.200106+0000","last_fullsized":"2023-11-28T11:17:50.200106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:32.183119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965277+0000","last_change":"2023-11-28T11:17:49.220560+0000","last_active":"2023-11-28T11:17:50.965277+0000","last_peered":"2023-11-28T11:17:50.965277+0000","last_clean":"2023-11-28T11:17:50.965277+0000","last_became_active":"2023-11-28T11:17:49.220013+0000","last_became_peered":"2023-11-28T11:17:49.220013+0000","last_unstale":"2023-11-28T11:17:50.965277+0000","last_undegraded":"2023-11-28T11:17:50.965277+0000","last_fullsized":"2023-11-28T11:17:50.965277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199285+0000","last_change":"2023-11-28T11:17:49.207080+0000","last_active":"2023-11-28T11:17:50.199285+0000","last_peered":"2023-11-28T11:17:50.199285+0000","last_clean":"2023-11-28T11:17:50.199285+0000","last_became_active":"2023-11-28T11:17:49.206569+0000","last_became_peered":"2023-11-28T11:17:49.206569+0000","last_unstale":"2023-11-28T11:17:50.199285+0000","last_undegraded":"2023-11-28T11:17:50.199285+0000","last_fullsized":"2023-11-28T11:17:50.199285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:19.859853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972845+0000","last_change":"2023-11-28T11:17:49.214151+0000","last_active":"2023-11-28T11:17:50.972845+0000","last_peered":"2023-11-28T11:17:50.972845+0000","last_clean":"2023-11-28T11:17:50.972845+0000","last_became_active":"2023-11-28T11:17:49.213623+0000","last_became_peered":"2023-11-28T11:17:49.213623+0000","last_unstale":"2023-11-28T11:17:50.972845+0000","last_undegraded":"2023-11-28T11:17:50.972845+0000","last_fullsized":"2023-11-28T11:17:50.972845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199866+0000","last_change":"2023-11-28T11:17:49.223078+0000","last_active":"2023-11-28T11:17:50.199866+0000","last_peered":"2023-11-28T11:17:50.199866+0000","last_clean":"2023-11-28T11:17:50.199866+0000","last_became_active":"2023-11-28T11:17:49.221852+0000","last_became_peered":"2023-11-28T11:17:49.221852+0000","last_unstale":"2023-11-28T11:17:50.199866+0000","last_undegraded":"2023-11-28T11:17:50.199866+0000","last_fullsized":"2023-11-28T11:17:50.199866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:47.568223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200736+0000","last_change":"2023-11-28T11:17:49.223486+0000","last_active":"2023-11-28T11:17:50.200736+0000","last_peered":"2023-11-28T11:17:50.200736+0000","last_clean":"2023-11-28T11:17:50.200736+0000","last_became_active":"2023-11-28T11:17:49.221471+0000","last_became_peered":"2023-11-28T11:17:49.221471+0000","last_unstale":"2023-11-28T11:17:50.200736+0000","last_undegraded":"2023-11-28T11:17:50.200736+0000","last_fullsized":"2023-11-28T11:17:50.200736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.963990+0000","last_change":"2023-11-28T11:17:49.208546+0000","last_active":"2023-11-28T11:17:50.963990+0000","last_peered":"2023-11-28T11:17:50.963990+0000","last_clean":"2023-11-28T11:17:50.963990+0000","last_became_active":"2023-11-28T11:17:49.208144+0000","last_became_peered":"2023-11-28T11:17:49.208144+0000","last_unstale":"2023-11-28T11:17:50.963990+0000","last_undegraded":"2023-11-28T11:17:50.963990+0000","last_fullsized":"2023-11-28T11:17:50.963990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.963973+0000","last_change":"2023-11-28T11:17:49.210677+0000","last_active":"2023-11-28T11:17:50.963973+0000","last_peered":"2023-11-28T11:17:50.963973+0000","last_clean":"2023-11-28T11:17:50.963973+0000","last_became_active":"2023-11-28T11:17:49.210425+0000","last_became_peered":"2023-11-28T11:17:49.210425+0000","last_unstale":"2023-11-28T11:17:50.963973+0000","last_undegraded":"2023-11-28T11:17:50.963973+0000","last_fullsized":"2023-11-28T11:17:50.963973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199673+0000","last_change":"2023-11-28T11:17:49.207731+0000","last_active":"2023-11-28T11:17:50.199673+0000","last_peered":"2023-11-28T11:17:50.199673+0000","last_clean":"2023-11-28T11:17:50.199673+0000","last_became_active":"2023-11-28T11:17:49.207411+0000","last_became_peered":"2023-11-28T11:17:49.207411+0000","last_unstale":"2023-11-28T11:17:50.199673+0000","last_undegraded":"2023-11-28T11:17:50.199673+0000","last_fullsized":"2023-11-28T11:17:50.199673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:57.387438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965314+0000","last_change":"2023-11-28T11:17:49.220742+0000","last_active":"2023-11-28T11:17:50.965314+0000","last_peered":"2023-11-28T11:17:50.965314+0000","last_clean":"2023-11-28T11:17:50.965314+0000","last_became_active":"2023-11-28T11:17:49.220406+0000","last_became_peered":"2023-11-28T11:17:49.220406+0000","last_unstale":"2023-11-28T11:17:50.965314+0000","last_undegraded":"2023-11-28T11:17:50.965314+0000","last_fullsized":"2023-11-28T11:17:50.965314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199468+0000","last_change":"2023-11-28T11:17:49.205686+0000","last_active":"2023-11-28T11:17:50.199468+0000","last_peered":"2023-11-28T11:17:50.199468+0000","last_clean":"2023-11-28T11:17:50.199468+0000","last_became_active":"2023-11-28T11:17:49.205310+0000","last_became_peered":"2023-11-28T11:17:49.205310+0000","last_unstale":"2023-11-28T11:17:50.199468+0000","last_undegraded":"2023-11-28T11:17:50.199468+0000","last_fullsized":"2023-11-28T11:17:50.199468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:23.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972853+0000","last_change":"2023-11-28T11:17:49.213934+0000","last_active":"2023-11-28T11:17:50.972853+0000","last_peered":"2023-11-28T11:17:50.972853+0000","last_clean":"2023-11-28T11:17:50.972853+0000","last_became_active":"2023-11-28T11:17:49.213324+0000","last_became_peered":"2023-11-28T11:17:49.213324+0000","last_unstale":"2023-11-28T11:17:50.972853+0000","last_undegraded":"2023-11-28T11:17:50.972853+0000","last_fullsized":"2023-11-28T11:17:50.972853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200680+0000","last_change":"2023-11-28T11:17:49.207509+0000","last_active":"2023-11-28T11:17:50.200680+0000","last_peered":"2023-11-28T11:17:50.200680+0000","last_clean":"2023-11-28T11:17:50.200680+0000","last_became_active":"2023-11-28T11:17:49.206337+0000","last_became_peered":"2023-11-28T11:17:49.206337+0000","last_unstale":"2023-11-28T11:17:50.200680+0000","last_undegraded":"2023-11-28T11:17:50.200680+0000","last_fullsized":"2023-11-28T11:17:50.200680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964010+0000","last_change":"2023-11-28T11:17:49.213657+0000","last_active":"2023-11-28T11:17:50.964010+0000","last_peered":"2023-11-28T11:17:50.964010+0000","last_clean":"2023-11-28T11:17:50.964010+0000","last_became_active":"2023-11-28T11:17:49.213294+0000","last_became_peered":"2023-11-28T11:17:49.213294+0000","last_unstale":"2023-11-28T11:17:50.964010+0000","last_undegraded":"2023-11-28T11:17:50.964010+0000","last_fullsized":"2023-11-28T11:17:50.964010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:50.362636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200525+0000","last_change":"2023-11-28T11:17:49.211522+0000","last_active":"2023-11-28T11:17:50.200525+0000","last_peered":"2023-11-28T11:17:50.200525+0000","last_clean":"2023-11-28T11:17:50.200525+0000","last_became_active":"2023-11-28T11:17:49.211218+0000","last_became_peered":"2023-11-28T11:17:49.211218+0000","last_unstale":"2023-11-28T11:17:50.200525+0000","last_undegraded":"2023-11-28T11:17:50.200525+0000","last_fullsized":"2023-11-28T11:17:50.200525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:54.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965353+0000","last_change":"2023-11-28T11:17:49.223624+0000","last_active":"2023-11-28T11:17:50.965353+0000","last_peered":"2023-11-28T11:17:50.965353+0000","last_clean":"2023-11-28T11:17:50.965353+0000","last_became_active":"2023-11-28T11:17:49.222927+0000","last_became_peered":"2023-11-28T11:17:49.222927+0000","last_unstale":"2023-11-28T11:17:50.965353+0000","last_undegraded":"2023-11-28T11:17:50.965353+0000","last_fullsized":"2023-11-28T11:17:50.965353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972725+0000","last_change":"2023-11-28T11:17:49.225717+0000","last_active":"2023-11-28T11:17:50.972725+0000","last_peered":"2023-11-28T11:17:50.972725+0000","last_clean":"2023-11-28T11:17:50.972725+0000","last_became_active":"2023-11-28T11:17:49.225457+0000","last_became_peered":"2023-11-28T11:17:49.225457+0000","last_unstale":"2023-11-28T11:17:50.972725+0000","last_undegraded":"2023-11-28T11:17:50.972725+0000","last_fullsized":"2023-11-28T11:17:50.972725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970424+0000","last_change":"2023-11-28T11:17:49.214153+0000","last_active":"2023-11-28T11:17:50.970424+0000","last_peered":"2023-11-28T11:17:50.970424+0000","last_clean":"2023-11-28T11:17:50.970424+0000","last_became_active":"2023-11-28T11:17:49.213231+0000","last_became_peered":"2023-11-28T11:17:49.213231+0000","last_unstale":"2023-11-28T11:17:50.970424+0000","last_undegraded":"2023-11-28T11:17:50.970424+0000","last_fullsized":"2023-11-28T11:17:50.970424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965242+0000","last_change":"2023-11-28T11:17:49.219658+0000","last_active":"2023-11-28T11:17:50.965242+0000","last_peered":"2023-11-28T11:17:50.965242+0000","last_clean":"2023-11-28T11:17:50.965242+0000","last_became_active":"2023-11-28T11:17:49.219357+0000","last_became_peered":"2023-11-28T11:17:49.219357+0000","last_unstale":"2023-11-28T11:17:50.965242+0000","last_undegraded":"2023-11-28T11:17:50.965242+0000","last_fullsized":"2023-11-28T11:17:50.965242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198788+0000","last_change":"2023-11-28T11:17:49.224287+0000","last_active":"2023-11-28T11:17:50.198788+0000","last_peered":"2023-11-28T11:17:50.198788+0000","last_clean":"2023-11-28T11:17:50.198788+0000","last_became_active":"2023-11-28T11:17:49.223202+0000","last_became_peered":"2023-11-28T11:17:49.223202+0000","last_unstale":"2023-11-28T11:17:50.198788+0000","last_undegraded":"2023-11-28T11:17:50.198788+0000","last_fullsized":"2023-11-28T11:17:50.198788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198664+0000","last_change":"2023-11-28T11:17:49.223865+0000","last_active":"2023-11-28T11:17:50.198664+0000","last_peered":"2023-11-28T11:17:50.198664+0000","last_clean":"2023-11-28T11:17:50.198664+0000","last_became_active":"2023-11-28T11:17:49.223609+0000","last_became_peered":"2023-11-28T11:17:49.223609+0000","last_unstale":"2023-11-28T11:17:50.198664+0000","last_undegraded":"2023-11-28T11:17:50.198664+0000","last_fullsized":"2023-11-28T11:17:50.198664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199744+0000","last_change":"2023-11-28T11:17:49.223914+0000","last_active":"2023-11-28T11:17:50.199744+0000","last_peered":"2023-11-28T11:17:50.199744+0000","last_clean":"2023-11-28T11:17:50.199744+0000","last_became_active":"2023-11-28T11:17:49.222584+0000","last_became_peered":"2023-11-28T11:17:49.222584+0000","last_unstale":"2023-11-28T11:17:50.199744+0000","last_undegraded":"2023-11-28T11:17:50.199744+0000","last_fullsized":"2023-11-28T11:17:50.199744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:52.447012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200227+0000","last_change":"2023-11-28T11:17:48.699811+0000","last_active":"2023-11-28T11:17:50.200227+0000","last_peered":"2023-11-28T11:17:50.200227+0000","last_clean":"2023-11-28T11:17:50.200227+0000","last_became_active":"2023-11-28T11:17:48.699476+0000","last_became_peered":"2023-11-28T11:17:48.699476+0000","last_unstale":"2023-11-28T11:17:50.200227+0000","last_undegraded":"2023-11-28T11:17:50.200227+0000","last_fullsized":"2023-11-28T11:17:50.200227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970498+0000","last_change":"2023-11-28T11:17:49.209310+0000","last_active":"2023-11-28T11:17:50.970498+0000","last_peered":"2023-11-28T11:17:50.970498+0000","last_clean":"2023-11-28T11:17:50.970498+0000","last_became_active":"2023-11-28T11:17:49.209040+0000","last_became_peered":"2023-11-28T11:17:49.209040+0000","last_unstale":"2023-11-28T11:17:50.970498+0000","last_undegraded":"2023-11-28T11:17:50.970498+0000","last_fullsized":"2023-11-28T11:17:50.970498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198161+0000","last_change":"2023-11-28T11:17:48.685181+0000","last_active":"2023-11-28T11:17:50.198161+0000","last_peered":"2023-11-28T11:17:50.198161+0000","last_clean":"2023-11-28T11:17:50.198161+0000","last_became_active":"2023-11-28T11:17:48.684843+0000","last_became_peered":"2023-11-28T11:17:48.684843+0000","last_unstale":"2023-11-28T11:17:50.198161+0000","last_undegraded":"2023-11-28T11:17:50.198161+0000","last_fullsized":"2023-11-28T11:17:50.198161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972795+0000","last_change":"2023-11-28T11:17:49.224963+0000","last_active":"2023-11-28T11:17:50.972795+0000","last_peered":"2023-11-28T11:17:50.972795+0000","last_clean":"2023-11-28T11:17:50.972795+0000","last_became_active":"2023-11-28T11:17:49.224679+0000","last_became_peered":"2023-11-28T11:17:49.224679+0000","last_unstale":"2023-11-28T11:17:50.972795+0000","last_undegraded":"2023-11-28T11:17:50.972795+0000","last_fullsized":"2023-11-28T11:17:50.972795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970396+0000","last_change":"2023-11-28T11:17:48.690687+0000","last_active":"2023-11-28T11:17:50.970396+0000","last_peered":"2023-11-28T11:17:50.970396+0000","last_clean":"2023-11-28T11:17:50.970396+0000","last_became_active":"2023-11-28T11:17:48.690509+0000","last_became_peered":"2023-11-28T11:17:48.690509+0000","last_unstale":"2023-11-28T11:17:50.970396+0000","last_undegraded":"2023-11-28T11:17:50.970396+0000","last_fullsized":"2023-11-28T11:17:50.970396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967085+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.967085+0000","last_peered":"2023-11-28T11:17:50.967085+0000","last_clean":"2023-11-28T11:17:50.967085+0000","last_became_active":"2023-11-28T11:17:49.215446+0000","last_became_peered":"2023-11-28T11:17:49.215446+0000","last_unstale":"2023-11-28T11:17:50.967085+0000","last_undegraded":"2023-11-28T11:17:50.967085+0000","last_fullsized":"2023-11-28T11:17:50.967085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972862+0000","last_change":"2023-11-28T11:17:48.689522+0000","last_active":"2023-11-28T11:17:50.972862+0000","last_peered":"2023-11-28T11:17:50.972862+0000","last_clean":"2023-11-28T11:17:50.972862+0000","last_became_active":"2023-11-28T11:17:48.689211+0000","last_became_peered":"2023-11-28T11:17:48.689211+0000","last_unstale":"2023-11-28T11:17:50.972862+0000","last_undegraded":"2023-11-28T11:17:50.972862+0000","last_fullsized":"2023-11-28T11:17:50.972862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970382+0000","last_change":"2023-11-28T11:17:49.222548+0000","last_active":"2023-11-28T11:17:50.970382+0000","last_peered":"2023-11-28T11:17:50.970382+0000","last_clean":"2023-11-28T11:17:50.970382+0000","last_became_active":"2023-11-28T11:17:49.222341+0000","last_became_peered":"2023-11-28T11:17:49.222341+0000","last_unstale":"2023-11-28T11:17:50.970382+0000","last_undegraded":"2023-11-28T11:17:50.970382+0000","last_fullsized":"2023-11-28T11:17:50.970382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199728+0000","last_change":"2023-11-28T11:17:48.696385+0000","last_active":"2023-11-28T11:17:50.199728+0000","last_peered":"2023-11-28T11:17:50.199728+0000","last_clean":"2023-11-28T11:17:50.199728+0000","last_became_active":"2023-11-28T11:17:48.696028+0000","last_became_peered":"2023-11-28T11:17:48.696028+0000","last_unstale":"2023-11-28T11:17:50.199728+0000","last_undegraded":"2023-11-28T11:17:50.199728+0000","last_fullsized":"2023-11-28T11:17:50.199728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:49.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970434+0000","last_change":"2023-11-28T11:17:49.210958+0000","last_active":"2023-11-28T11:17:50.970434+0000","last_peered":"2023-11-28T11:17:50.970434+0000","last_clean":"2023-11-28T11:17:50.970434+0000","last_became_active":"2023-11-28T11:17:49.210651+0000","last_became_peered":"2023-11-28T11:17:49.210651+0000","last_unstale":"2023-11-28T11:17:50.970434+0000","last_undegraded":"2023-11-28T11:17:50.970434+0000","last_fullsized":"2023-11-28T11:17:50.970434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965376+0000","last_change":"2023-11-28T11:17:48.697605+0000","last_active":"2023-11-28T11:17:50.965376+0000","last_peered":"2023-11-28T11:17:50.965376+0000","last_clean":"2023-11-28T11:17:50.965376+0000","last_became_active":"2023-11-28T11:17:48.696308+0000","last_became_peered":"2023-11-28T11:17:48.696308+0000","last_unstale":"2023-11-28T11:17:50.965376+0000","last_undegraded":"2023-11-28T11:17:50.965376+0000","last_fullsized":"2023-11-28T11:17:50.965376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:46.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198244+0000","last_change":"2023-11-28T11:17:49.213923+0000","last_active":"2023-11-28T11:17:50.198244+0000","last_peered":"2023-11-28T11:17:50.198244+0000","last_clean":"2023-11-28T11:17:50.198244+0000","last_became_active":"2023-11-28T11:17:49.213693+0000","last_became_peered":"2023-11-28T11:17:49.213693+0000","last_unstale":"2023-11-28T11:17:50.198244+0000","last_undegraded":"2023-11-28T11:17:50.198244+0000","last_fullsized":"2023-11-28T11:17:50.198244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200624+0000","last_change":"2023-11-28T11:17:48.693861+0000","last_active":"2023-11-28T11:17:50.200624+0000","last_peered":"2023-11-28T11:17:50.200624+0000","last_clean":"2023-11-28T11:17:50.200624+0000","last_became_active":"2023-11-28T11:17:48.693491+0000","last_became_peered":"2023-11-28T11:17:48.693491+0000","last_unstale":"2023-11-28T11:17:50.200624+0000","last_undegraded":"2023-11-28T11:17:50.200624+0000","last_fullsized":"2023-11-28T11:17:50.200624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200248+0000","last_change":"2023-11-28T11:17:49.213236+0000","last_active":"2023-11-28T11:17:50.200248+0000","last_peered":"2023-11-28T11:17:50.200248+0000","last_clean":"2023-11-28T11:17:50.200248+0000","last_became_active":"2023-11-28T11:17:49.211752+0000","last_became_peered":"2023-11-28T11:17:49.211752+0000","last_unstale":"2023-11-28T11:17:50.200248+0000","last_undegraded":"2023-11-28T11:17:50.200248+0000","last_fullsized":"2023-11-28T11:17:50.200248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:40.807651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965333+0000","last_change":"2023-11-28T11:17:48.702529+0000","last_active":"2023-11-28T11:17:50.965333+0000","last_peered":"2023-11-28T11:17:50.965333+0000","last_clean":"2023-11-28T11:17:50.965333+0000","last_became_active":"2023-11-28T11:17:48.702137+0000","last_became_peered":"2023-11-28T11:17:48.702137+0000","last_unstale":"2023-11-28T11:17:50.965333+0000","last_undegraded":"2023-11-28T11:17:50.965333+0000","last_fullsized":"2023-11-28T11:17:50.965333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:49.562684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970489+0000","last_change":"2023-11-28T11:17:49.223453+0000","last_active":"2023-11-28T11:17:50.970489+0000","last_peered":"2023-11-28T11:17:50.970489+0000","last_clean":"2023-11-28T11:17:50.970489+0000","last_became_active":"2023-11-28T11:17:49.222771+0000","last_became_peered":"2023-11-28T11:17:49.222771+0000","last_unstale":"2023-11-28T11:17:50.970489+0000","last_undegraded":"2023-11-28T11:17:50.970489+0000","last_fullsized":"2023-11-28T11:17:50.970489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972714+0000","last_change":"2023-11-28T11:17:48.689983+0000","last_active":"2023-11-28T11:17:50.972714+0000","last_peered":"2023-11-28T11:17:50.972714+0000","last_clean":"2023-11-28T11:17:50.972714+0000","last_became_active":"2023-11-28T11:17:48.688579+0000","last_became_peered":"2023-11-28T11:17:48.688579+0000","last_unstale":"2023-11-28T11:17:50.972714+0000","last_undegraded":"2023-11-28T11:17:50.972714+0000","last_fullsized":"2023-11-28T11:17:50.972714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199828+0000","last_change":"2023-11-28T11:17:49.219334+0000","last_active":"2023-11-28T11:17:50.199828+0000","last_peered":"2023-11-28T11:17:50.199828+0000","last_clean":"2023-11-28T11:17:50.199828+0000","last_became_active":"2023-11-28T11:17:49.218912+0000","last_became_peered":"2023-11-28T11:17:49.218912+0000","last_unstale":"2023-11-28T11:17:50.199828+0000","last_undegraded":"2023-11-28T11:17:50.199828+0000","last_fullsized":"2023-11-28T11:17:50.199828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:29.752626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965296+0000","last_change":"2023-11-28T11:17:48.702427+0000","last_active":"2023-11-28T11:17:50.965296+0000","last_peered":"2023-11-28T11:17:50.965296+0000","last_clean":"2023-11-28T11:17:50.965296+0000","last_became_active":"2023-11-28T11:17:48.702118+0000","last_became_peered":"2023-11-28T11:17:48.702118+0000","last_unstale":"2023-11-28T11:17:50.965296+0000","last_undegraded":"2023-11-28T11:17:50.965296+0000","last_fullsized":"2023-11-28T11:17:50.965296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:37.239541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972785+0000","last_change":"2023-11-28T11:17:49.209958+0000","last_active":"2023-11-28T11:17:50.972785+0000","last_peered":"2023-11-28T11:17:50.972785+0000","last_clean":"2023-11-28T11:17:50.972785+0000","last_became_active":"2023-11-28T11:17:49.209616+0000","last_became_peered":"2023-11-28T11:17:49.209616+0000","last_unstale":"2023-11-28T11:17:50.972785+0000","last_undegraded":"2023-11-28T11:17:50.972785+0000","last_fullsized":"2023-11-28T11:17:50.972785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966956+0000","last_change":"2023-11-28T11:17:48.696749+0000","last_active":"2023-11-28T11:17:50.966956+0000","last_peered":"2023-11-28T11:17:50.966956+0000","last_clean":"2023-11-28T11:17:50.966956+0000","last_became_active":"2023-11-28T11:17:48.695400+0000","last_became_peered":"2023-11-28T11:17:48.695400+0000","last_unstale":"2023-11-28T11:17:50.966956+0000","last_undegraded":"2023-11-28T11:17:50.966956+0000","last_fullsized":"2023-11-28T11:17:50.966956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970533+0000","last_change":"2023-11-28T11:17:49.209165+0000","last_active":"2023-11-28T11:17:50.970533+0000","last_peered":"2023-11-28T11:17:50.970533+0000","last_clean":"2023-11-28T11:17:50.970533+0000","last_became_active":"2023-11-28T11:17:49.208916+0000","last_became_peered":"2023-11-28T11:17:49.208916+0000","last_unstale":"2023-11-28T11:17:50.970533+0000","last_undegraded":"2023-11-28T11:17:50.970533+0000","last_fullsized":"2023-11-28T11:17:50.970533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965478+0000","last_change":"2023-11-28T11:17:48.700934+0000","last_active":"2023-11-28T11:17:50.965478+0000","last_peered":"2023-11-28T11:17:50.965478+0000","last_clean":"2023-11-28T11:17:50.965478+0000","last_became_active":"2023-11-28T11:17:48.700549+0000","last_became_peered":"2023-11-28T11:17:48.700549+0000","last_unstale":"2023-11-28T11:17:50.965478+0000","last_undegraded":"2023-11-28T11:17:50.965478+0000","last_fullsized":"2023-11-28T11:17:50.965478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:40.639240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199331+0000","last_change":"2023-11-28T11:17:49.218909+0000","last_active":"2023-11-28T11:17:50.199331+0000","last_peered":"2023-11-28T11:17:50.199331+0000","last_clean":"2023-11-28T11:17:50.199331+0000","last_became_active":"2023-11-28T11:17:49.218582+0000","last_became_peered":"2023-11-28T11:17:49.218582+0000","last_unstale":"2023-11-28T11:17:50.199331+0000","last_undegraded":"2023-11-28T11:17:50.199331+0000","last_fullsized":"2023-11-28T11:17:50.199331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:25.463820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965454+0000","last_change":"2023-11-28T11:17:48.698986+0000","last_active":"2023-11-28T11:17:50.965454+0000","last_peered":"2023-11-28T11:17:50.965454+0000","last_clean":"2023-11-28T11:17:50.965454+0000","last_became_active":"2023-11-28T11:17:48.698594+0000","last_became_peered":"2023-11-28T11:17:48.698594+0000","last_unstale":"2023-11-28T11:17:50.965454+0000","last_undegraded":"2023-11-28T11:17:50.965454+0000","last_fullsized":"2023-11-28T11:17:50.965454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:19.087105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199992+0000","last_change":"2023-11-28T11:17:49.213092+0000","last_active":"2023-11-28T11:17:50.199992+0000","last_peered":"2023-11-28T11:17:50.199992+0000","last_clean":"2023-11-28T11:17:50.199992+0000","last_became_active":"2023-11-28T11:17:49.211870+0000","last_became_peered":"2023-11-28T11:17:49.211870+0000","last_unstale":"2023-11-28T11:17:50.199992+0000","last_undegraded":"2023-11-28T11:17:50.199992+0000","last_fullsized":"2023-11-28T11:17:50.199992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:41.645456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965432+0000","last_change":"2023-11-28T11:17:48.695292+0000","last_active":"2023-11-28T11:17:50.965432+0000","last_peered":"2023-11-28T11:17:50.965432+0000","last_clean":"2023-11-28T11:17:50.965432+0000","last_became_active":"2023-11-28T11:17:48.694985+0000","last_became_peered":"2023-11-28T11:17:48.694985+0000","last_unstale":"2023-11-28T11:17:50.965432+0000","last_undegraded":"2023-11-28T11:17:50.965432+0000","last_fullsized":"2023-11-28T11:17:50.965432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:56.077529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966999+0000","last_change":"2023-11-28T11:17:49.226629+0000","last_active":"2023-11-28T11:17:50.966999+0000","last_peered":"2023-11-28T11:17:50.966999+0000","last_clean":"2023-11-28T11:17:50.966999+0000","last_became_active":"2023-11-28T11:17:49.226347+0000","last_became_peered":"2023-11-28T11:17:49.226347+0000","last_unstale":"2023-11-28T11:17:50.966999+0000","last_undegraded":"2023-11-28T11:17:50.966999+0000","last_fullsized":"2023-11-28T11:17:50.966999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199618+0000","last_change":"2023-11-28T11:17:48.692881+0000","last_active":"2023-11-28T11:17:50.199618+0000","last_peered":"2023-11-28T11:17:50.199618+0000","last_clean":"2023-11-28T11:17:50.199618+0000","last_became_active":"2023-11-28T11:17:48.692426+0000","last_became_peered":"2023-11-28T11:17:48.692426+0000","last_unstale":"2023-11-28T11:17:50.199618+0000","last_undegraded":"2023-11-28T11:17:50.199618+0000","last_fullsized":"2023-11-28T11:17:50.199618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:23.247523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972735+0000","last_change":"2023-11-28T11:17:49.221185+0000","last_active":"2023-11-28T11:17:50.972735+0000","last_peered":"2023-11-28T11:17:50.972735+0000","last_clean":"2023-11-28T11:17:50.972735+0000","last_became_active":"2023-11-28T11:17:49.220927+0000","last_became_peered":"2023-11-28T11:17:49.220927+0000","last_unstale":"2023-11-28T11:17:50.972735+0000","last_undegraded":"2023-11-28T11:17:50.972735+0000","last_fullsized":"2023-11-28T11:17:50.972735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970455+0000","last_change":"2023-11-28T11:17:48.698335+0000","last_active":"2023-11-28T11:17:50.970455+0000","last_peered":"2023-11-28T11:17:50.970455+0000","last_clean":"2023-11-28T11:17:50.970455+0000","last_became_active":"2023-11-28T11:17:48.698032+0000","last_became_peered":"2023-11-28T11:17:48.698032+0000","last_unstale":"2023-11-28T11:17:50.970455+0000","last_undegraded":"2023-11-28T11:17:50.970455+0000","last_fullsized":"2023-11-28T11:17:50.970455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200609+0000","last_change":"2023-11-28T11:17:49.207155+0000","last_active":"2023-11-28T11:17:50.200609+0000","last_peered":"2023-11-28T11:17:50.200609+0000","last_clean":"2023-11-28T11:17:50.200609+0000","last_became_active":"2023-11-28T11:17:49.206800+0000","last_became_peered":"2023-11-28T11:17:49.206800+0000","last_unstale":"2023-11-28T11:17:50.200609+0000","last_undegraded":"2023-11-28T11:17:50.200609+0000","last_fullsized":"2023-11-28T11:17:50.200609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966985+0000","last_change":"2023-11-28T11:17:48.702363+0000","last_active":"2023-11-28T11:17:50.966985+0000","last_peered":"2023-11-28T11:17:50.966985+0000","last_clean":"2023-11-28T11:17:50.966985+0000","last_became_active":"2023-11-28T11:17:48.701778+0000","last_became_peered":"2023-11-28T11:17:48.701778+0000","last_unstale":"2023-11-28T11:17:50.966985+0000","last_undegraded":"2023-11-28T11:17:50.966985+0000","last_fullsized":"2023-11-28T11:17:50.966985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200788+0000","last_change":"2023-11-28T11:17:49.205845+0000","last_active":"2023-11-28T11:17:50.200788+0000","last_peered":"2023-11-28T11:17:50.200788+0000","last_clean":"2023-11-28T11:17:50.200788+0000","last_became_active":"2023-11-28T11:17:49.205302+0000","last_became_peered":"2023-11-28T11:17:49.205302+0000","last_unstale":"2023-11-28T11:17:50.200788+0000","last_undegraded":"2023-11-28T11:17:50.200788+0000","last_fullsized":"2023-11-28T11:17:50.200788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200199+0000","last_change":"2023-11-28T11:17:48.699968+0000","last_active":"2023-11-28T11:17:50.200199+0000","last_peered":"2023-11-28T11:17:50.200199+0000","last_clean":"2023-11-28T11:17:50.200199+0000","last_became_active":"2023-11-28T11:17:48.697897+0000","last_became_peered":"2023-11-28T11:17:48.697897+0000","last_unstale":"2023-11-28T11:17:50.200199+0000","last_undegraded":"2023-11-28T11:17:50.200199+0000","last_fullsized":"2023-11-28T11:17:50.200199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200758+0000","last_change":"2023-11-28T11:17:49.210939+0000","last_active":"2023-11-28T11:17:50.200758+0000","last_peered":"2023-11-28T11:17:50.200758+0000","last_clean":"2023-11-28T11:17:50.200758+0000","last_became_active":"2023-11-28T11:17:49.210585+0000","last_became_peered":"2023-11-28T11:17:49.210585+0000","last_unstale":"2023-11-28T11:17:50.200758+0000","last_undegraded":"2023-11-28T11:17:50.200758+0000","last_fullsized":"2023-11-28T11:17:50.200758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:17.930855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970354+0000","last_change":"2023-11-28T11:17:48.687630+0000","last_active":"2023-11-28T11:17:50.970354+0000","last_peered":"2023-11-28T11:17:50.970354+0000","last_clean":"2023-11-28T11:17:50.970354+0000","last_became_active":"2023-11-28T11:17:48.685494+0000","last_became_peered":"2023-11-28T11:17:48.685494+0000","last_unstale":"2023-11-28T11:17:50.970354+0000","last_undegraded":"2023-11-28T11:17:50.970354+0000","last_fullsized":"2023-11-28T11:17:50.970354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198866+0000","last_change":"2023-11-28T11:17:49.223631+0000","last_active":"2023-11-28T11:17:50.198866+0000","last_peered":"2023-11-28T11:17:50.198866+0000","last_clean":"2023-11-28T11:17:50.198866+0000","last_became_active":"2023-11-28T11:17:49.223321+0000","last_became_peered":"2023-11-28T11:17:49.223321+0000","last_unstale":"2023-11-28T11:17:50.198866+0000","last_undegraded":"2023-11-28T11:17:50.198866+0000","last_fullsized":"2023-11-28T11:17:50.198866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.195384+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:17:49.195384+0000","last_peered":"2023-11-28T11:17:49.195384+0000","last_clean":"2023-11-28T11:17:49.195384+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:17:49.195384+0000","last_undegraded":"2023-11-28T11:17:49.195384+0000","last_fullsized":"2023-11-28T11:17:49.195384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199900+0000","last_change":"2023-11-28T11:17:48.691483+0000","last_active":"2023-11-28T11:17:50.199900+0000","last_peered":"2023-11-28T11:17:50.199900+0000","last_clean":"2023-11-28T11:17:50.199900+0000","last_became_active":"2023-11-28T11:17:48.691194+0000","last_became_peered":"2023-11-28T11:17:48.691194+0000","last_unstale":"2023-11-28T11:17:50.199900+0000","last_undegraded":"2023-11-28T11:17:50.199900+0000","last_fullsized":"2023-11-28T11:17:50.199900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:43.938903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198294+0000","last_change":"2023-11-28T11:17:49.217104+0000","last_active":"2023-11-28T11:17:50.198294+0000","last_peered":"2023-11-28T11:17:50.198294+0000","last_clean":"2023-11-28T11:17:50.198294+0000","last_became_active":"2023-11-28T11:17:49.216883+0000","last_became_peered":"2023-11-28T11:17:49.216883+0000","last_unstale":"2023-11-28T11:17:50.198294+0000","last_undegraded":"2023-11-28T11:17:50.198294+0000","last_fullsized":"2023-11-28T11:17:50.198294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202595+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:17:49.202595+0000","last_peered":"2023-11-28T11:17:49.202595+0000","last_clean":"2023-11-28T11:17:49.202595+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:17:49.202595+0000","last_undegraded":"2023-11-28T11:17:49.202595+0000","last_fullsized":"2023-11-28T11:17:49.202595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200374+0000","last_change":"2023-11-28T11:17:48.699726+0000","last_active":"2023-11-28T11:17:50.200374+0000","last_peered":"2023-11-28T11:17:50.200374+0000","last_clean":"2023-11-28T11:17:50.200374+0000","last_became_active":"2023-11-28T11:17:48.699409+0000","last_became_peered":"2023-11-28T11:17:48.699409+0000","last_unstale":"2023-11-28T11:17:50.200374+0000","last_undegraded":"2023-11-28T11:17:50.200374+0000","last_fullsized":"2023-11-28T11:17:50.200374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198558+0000","last_change":"2023-11-28T11:17:49.223120+0000","last_active":"2023-11-28T11:17:50.198558+0000","last_peered":"2023-11-28T11:17:50.198558+0000","last_clean":"2023-11-28T11:17:50.198558+0000","last_became_active":"2023-11-28T11:17:49.217684+0000","last_became_peered":"2023-11-28T11:17:49.217684+0000","last_unstale":"2023-11-28T11:17:50.198558+0000","last_undegraded":"2023-11-28T11:17:50.198558+0000","last_fullsized":"2023-11-28T11:17:50.198558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199240+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:17:49.199240+0000","last_peered":"2023-11-28T11:17:49.199240+0000","last_clean":"2023-11-28T11:17:49.199240+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:17:49.199240+0000","last_undegraded":"2023-11-28T11:17:49.199240+0000","last_fullsized":"2023-11-28T11:17:49.199240+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198094+0000","last_change":"2023-11-28T11:17:48.695369+0000","last_active":"2023-11-28T11:17:50.198094+0000","last_peered":"2023-11-28T11:17:50.198094+0000","last_clean":"2023-11-28T11:17:50.198094+0000","last_became_active":"2023-11-28T11:17:48.695065+0000","last_became_peered":"2023-11-28T11:17:48.695065+0000","last_unstale":"2023-11-28T11:17:50.198094+0000","last_undegraded":"2023-11-28T11:17:50.198094+0000","last_fullsized":"2023-11-28T11:17:50.198094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198772+0000","last_change":"2023-11-28T11:17:49.217116+0000","last_active":"2023-11-28T11:17:50.198772+0000","last_peered":"2023-11-28T11:17:50.198772+0000","last_clean":"2023-11-28T11:17:50.198772+0000","last_became_active":"2023-11-28T11:17:49.216818+0000","last_became_peered":"2023-11-28T11:17:49.216818+0000","last_unstale":"2023-11-28T11:17:50.198772+0000","last_undegraded":"2023-11-28T11:17:50.198772+0000","last_fullsized":"2023-11-28T11:17:50.198772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202500+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:17:49.202500+0000","last_peered":"2023-11-28T11:17:49.202500+0000","last_clean":"2023-11-28T11:17:49.202500+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:17:49.202500+0000","last_undegraded":"2023-11-28T11:17:49.202500+0000","last_fullsized":"2023-11-28T11:17:49.202500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.198393+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:17:49.198393+0000","last_peered":"2023-11-28T11:17:49.198393+0000","last_clean":"2023-11-28T11:17:49.198393+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:17:49.198393+0000","last_undegraded":"2023-11-28T11:17:49.198393+0000","last_fullsized":"2023-11-28T11:17:49.198393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970443+0000","last_change":"2023-11-28T11:17:48.692995+0000","last_active":"2023-11-28T11:17:50.970443+0000","last_peered":"2023-11-28T11:17:50.970443+0000","last_clean":"2023-11-28T11:17:50.970443+0000","last_became_active":"2023-11-28T11:17:48.692619+0000","last_became_peered":"2023-11-28T11:17:48.692619+0000","last_unstale":"2023-11-28T11:17:50.970443+0000","last_undegraded":"2023-11-28T11:17:50.970443+0000","last_fullsized":"2023-11-28T11:17:50.970443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967024+0000","last_change":"2023-11-28T11:17:49.211592+0000","last_active":"2023-11-28T11:17:50.967024+0000","last_peered":"2023-11-28T11:17:50.967024+0000","last_clean":"2023-11-28T11:17:50.967024+0000","last_became_active":"2023-11-28T11:17:49.209606+0000","last_became_peered":"2023-11-28T11:17:49.209606+0000","last_unstale":"2023-11-28T11:17:50.967024+0000","last_undegraded":"2023-11-28T11:17:50.967024+0000","last_fullsized":"2023-11-28T11:17:50.967024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:47.444676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199814+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:17:49.199814+0000","last_peered":"2023-11-28T11:17:49.199814+0000","last_clean":"2023-11-28T11:17:49.199814+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:17:49.199814+0000","last_undegraded":"2023-11-28T11:17:49.199814+0000","last_fullsized":"2023-11-28T11:17:49.199814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972688+0000","last_change":"2023-11-28T11:17:48.696786+0000","last_active":"2023-11-28T11:17:50.972688+0000","last_peered":"2023-11-28T11:17:50.972688+0000","last_clean":"2023-11-28T11:17:50.972688+0000","last_became_active":"2023-11-28T11:17:48.696474+0000","last_became_peered":"2023-11-28T11:17:48.696474+0000","last_unstale":"2023-11-28T11:17:50.972688+0000","last_undegraded":"2023-11-28T11:17:50.972688+0000","last_fullsized":"2023-11-28T11:17:50.972688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972836+0000","last_change":"2023-11-28T11:17:49.219523+0000","last_active":"2023-11-28T11:17:50.972836+0000","last_peered":"2023-11-28T11:17:50.972836+0000","last_clean":"2023-11-28T11:17:50.972836+0000","last_became_active":"2023-11-28T11:17:49.219226+0000","last_became_peered":"2023-11-28T11:17:49.219226+0000","last_unstale":"2023-11-28T11:17:50.972836+0000","last_undegraded":"2023-11-28T11:17:50.972836+0000","last_fullsized":"2023-11-28T11:17:50.972836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199998+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:17:49.199998+0000","last_peered":"2023-11-28T11:17:49.199998+0000","last_clean":"2023-11-28T11:17:49.199998+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:17:49.199998+0000","last_undegraded":"2023-11-28T11:17:49.199998+0000","last_fullsized":"2023-11-28T11:17:49.199998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198637+0000","last_change":"2023-11-28T11:17:48.696130+0000","last_active":"2023-11-28T11:17:50.198637+0000","last_peered":"2023-11-28T11:17:50.198637+0000","last_clean":"2023-11-28T11:17:50.198637+0000","last_became_active":"2023-11-28T11:17:48.695765+0000","last_became_peered":"2023-11-28T11:17:48.695765+0000","last_unstale":"2023-11-28T11:17:50.198637+0000","last_undegraded":"2023-11-28T11:17:50.198637+0000","last_fullsized":"2023-11-28T11:17:50.198637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972753+0000","last_change":"2023-11-28T11:17:49.219689+0000","last_active":"2023-11-28T11:17:50.972753+0000","last_peered":"2023-11-28T11:17:50.972753+0000","last_clean":"2023-11-28T11:17:50.972753+0000","last_became_active":"2023-11-28T11:17:49.219446+0000","last_became_peered":"2023-11-28T11:17:49.219446+0000","last_unstale":"2023-11-28T11:17:50.972753+0000","last_undegraded":"2023-11-28T11:17:50.972753+0000","last_fullsized":"2023-11-28T11:17:50.972753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970522+0000","last_change":"2023-11-28T11:17:48.694653+0000","last_active":"2023-11-28T11:17:50.970522+0000","last_peered":"2023-11-28T11:17:50.970522+0000","last_clean":"2023-11-28T11:17:50.970522+0000","last_became_active":"2023-11-28T11:17:48.694360+0000","last_became_peered":"2023-11-28T11:17:48.694360+0000","last_unstale":"2023-11-28T11:17:50.970522+0000","last_undegraded":"2023-11-28T11:17:50.970522+0000","last_fullsized":"2023-11-28T11:17:50.970522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.194295+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:17:49.194295+0000","last_peered":"2023-11-28T11:17:49.194295+0000","last_clean":"2023-11-28T11:17:49.194295+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:17:49.194295+0000","last_undegraded":"2023-11-28T11:17:49.194295+0000","last_fullsized":"2023-11-28T11:17:49.194295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970337+0000","last_change":"2023-11-28T11:17:49.219150+0000","last_active":"2023-11-28T11:17:50.970337+0000","last_peered":"2023-11-28T11:17:50.970337+0000","last_clean":"2023-11-28T11:17:50.970337+0000","last_became_active":"2023-11-28T11:17:49.218887+0000","last_became_peered":"2023-11-28T11:17:49.218887+0000","last_unstale":"2023-11-28T11:17:50.970337+0000","last_undegraded":"2023-11-28T11:17:50.970337+0000","last_fullsized":"2023-11-28T11:17:50.970337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.204699+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:17:49.204699+0000","last_peered":"2023-11-28T11:17:49.204699+0000","last_clean":"2023-11-28T11:17:49.204699+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:17:49.204699+0000","last_undegraded":"2023-11-28T11:17:49.204699+0000","last_fullsized":"2023-11-28T11:17:49.204699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:51.413066+0000","last_change":"2023-11-28T11:17:48.688812+0000","last_active":"2023-11-28T11:17:51.413066+0000","last_peered":"2023-11-28T11:17:51.413066+0000","last_clean":"2023-11-28T11:17:51.413066+0000","last_became_active":"2023-11-28T11:17:48.688551+0000","last_became_peered":"2023-11-28T11:17:48.688551+0000","last_unstale":"2023-11-28T11:17:51.413066+0000","last_undegraded":"2023-11-28T11:17:51.413066+0000","last_fullsized":"2023-11-28T11:17:51.413066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200103+0000","last_change":"2023-11-28T11:17:49.207524+0000","last_active":"2023-11-28T11:17:50.200103+0000","last_peered":"2023-11-28T11:17:50.200103+0000","last_clean":"2023-11-28T11:17:50.200103+0000","last_became_active":"2023-11-28T11:17:49.207175+0000","last_became_peered":"2023-11-28T11:17:49.207175+0000","last_unstale":"2023-11-28T11:17:50.200103+0000","last_undegraded":"2023-11-28T11:17:50.200103+0000","last_fullsized":"2023-11-28T11:17:50.200103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:42.481462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972774+0000","last_change":"2023-11-28T11:17:48.691892+0000","last_active":"2023-11-28T11:17:50.972774+0000","last_peered":"2023-11-28T11:17:50.972774+0000","last_clean":"2023-11-28T11:17:50.972774+0000","last_became_active":"2023-11-28T11:17:48.691619+0000","last_became_peered":"2023-11-28T11:17:48.691619+0000","last_unstale":"2023-11-28T11:17:50.972774+0000","last_undegraded":"2023-11-28T11:17:50.972774+0000","last_fullsized":"2023-11-28T11:17:50.972774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964001+0000","last_change":"2023-11-28T11:17:49.230012+0000","last_active":"2023-11-28T11:17:50.964001+0000","last_peered":"2023-11-28T11:17:50.964001+0000","last_clean":"2023-11-28T11:17:50.964001+0000","last_became_active":"2023-11-28T11:17:49.229696+0000","last_became_peered":"2023-11-28T11:17:49.229696+0000","last_unstale":"2023-11-28T11:17:50.964001+0000","last_undegraded":"2023-11-28T11:17:50.964001+0000","last_fullsized":"2023-11-28T11:17:50.964001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199712+0000","last_change":"2023-11-28T11:17:48.699672+0000","last_active":"2023-11-28T11:17:50.199712+0000","last_peered":"2023-11-28T11:17:50.199712+0000","last_clean":"2023-11-28T11:17:50.199712+0000","last_became_active":"2023-11-28T11:17:48.699372+0000","last_became_peered":"2023-11-28T11:17:48.699372+0000","last_unstale":"2023-11-28T11:17:50.199712+0000","last_undegraded":"2023-11-28T11:17:50.199712+0000","last_fullsized":"2023-11-28T11:17:50.199712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:53.370355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200423+0000","last_change":"2023-11-28T11:17:49.212871+0000","last_active":"2023-11-28T11:17:50.200423+0000","last_peered":"2023-11-28T11:17:50.200423+0000","last_clean":"2023-11-28T11:17:50.200423+0000","last_became_active":"2023-11-28T11:17:49.212601+0000","last_became_peered":"2023-11-28T11:17:49.212601+0000","last_unstale":"2023-11-28T11:17:50.200423+0000","last_undegraded":"2023-11-28T11:17:50.200423+0000","last_fullsized":"2023-11-28T11:17:50.200423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199934+0000","last_change":"2023-11-28T11:17:48.697131+0000","last_active":"2023-11-28T11:17:50.199934+0000","last_peered":"2023-11-28T11:17:50.199934+0000","last_clean":"2023-11-28T11:17:50.199934+0000","last_became_active":"2023-11-28T11:17:48.696849+0000","last_became_peered":"2023-11-28T11:17:48.696849+0000","last_unstale":"2023-11-28T11:17:50.199934+0000","last_undegraded":"2023-11-28T11:17:50.199934+0000","last_fullsized":"2023-11-28T11:17:50.199934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:09.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200297+0000","last_change":"2023-11-28T11:17:49.216478+0000","last_active":"2023-11-28T11:17:50.200297+0000","last_peered":"2023-11-28T11:17:50.200297+0000","last_clean":"2023-11-28T11:17:50.200297+0000","last_became_active":"2023-11-28T11:17:49.215792+0000","last_became_peered":"2023-11-28T11:17:49.215792+0000","last_unstale":"2023-11-28T11:17:50.200297+0000","last_undegraded":"2023-11-28T11:17:50.200297+0000","last_fullsized":"2023-11-28T11:17:50.200297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970509+0000","last_change":"2023-11-28T11:17:48.692669+0000","last_active":"2023-11-28T11:17:50.970509+0000","last_peered":"2023-11-28T11:17:50.970509+0000","last_clean":"2023-11-28T11:17:50.970509+0000","last_became_active":"2023-11-28T11:17:48.691780+0000","last_became_peered":"2023-11-28T11:17:48.691780+0000","last_unstale":"2023-11-28T11:17:50.970509+0000","last_undegraded":"2023-11-28T11:17:50.970509+0000","last_fullsized":"2023-11-28T11:17:50.970509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970415+0000","last_change":"2023-11-28T11:17:49.220031+0000","last_active":"2023-11-28T11:17:50.970415+0000","last_peered":"2023-11-28T11:17:50.970415+0000","last_clean":"2023-11-28T11:17:50.970415+0000","last_became_active":"2023-11-28T11:17:49.219784+0000","last_became_peered":"2023-11-28T11:17:49.219784+0000","last_unstale":"2023-11-28T11:17:50.970415+0000","last_undegraded":"2023-11-28T11:17:50.970415+0000","last_fullsized":"2023-11-28T11:17:50.970415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972874+0000","last_change":"2023-11-28T11:17:48.689149+0000","last_active":"2023-11-28T11:17:50.972874+0000","last_peered":"2023-11-28T11:17:50.972874+0000","last_clean":"2023-11-28T11:17:50.972874+0000","last_became_active":"2023-11-28T11:17:48.688826+0000","last_became_peered":"2023-11-28T11:17:48.688826+0000","last_unstale":"2023-11-28T11:17:50.972874+0000","last_undegraded":"2023-11-28T11:17:50.972874+0000","last_fullsized":"2023-11-28T11:17:50.972874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199319+0000","last_change":"2023-11-28T11:17:49.208765+0000","last_active":"2023-11-28T11:17:50.199319+0000","last_peered":"2023-11-28T11:17:50.199319+0000","last_clean":"2023-11-28T11:17:50.199319+0000","last_became_active":"2023-11-28T11:17:49.208432+0000","last_became_peered":"2023-11-28T11:17:49.208432+0000","last_unstale":"2023-11-28T11:17:50.199319+0000","last_undegraded":"2023-11-28T11:17:50.199319+0000","last_fullsized":"2023-11-28T11:17:50.199319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966973+0000","last_change":"2023-11-28T11:17:49.210834+0000","last_active":"2023-11-28T11:17:50.966973+0000","last_peered":"2023-11-28T11:17:50.966973+0000","last_clean":"2023-11-28T11:17:50.966973+0000","last_became_active":"2023-11-28T11:17:49.208613+0000","last_became_peered":"2023-11-28T11:17:49.208613+0000","last_unstale":"2023-11-28T11:17:50.966973+0000","last_undegraded":"2023-11-28T11:17:50.966973+0000","last_fullsized":"2023-11-28T11:17:50.966973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970406+0000","last_change":"2023-11-28T11:17:49.223484+0000","last_active":"2023-11-28T11:17:50.970406+0000","last_peered":"2023-11-28T11:17:50.970406+0000","last_clean":"2023-11-28T11:17:50.970406+0000","last_became_active":"2023-11-28T11:17:49.222011+0000","last_became_peered":"2023-11-28T11:17:49.222011+0000","last_unstale":"2023-11-28T11:17:50.970406+0000","last_undegraded":"2023-11-28T11:17:50.970406+0000","last_fullsized":"2023-11-28T11:17:50.970406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200166+0000","last_change":"2023-11-28T11:17:49.216669+0000","last_active":"2023-11-28T11:17:50.200166+0000","last_peered":"2023-11-28T11:17:50.200166+0000","last_clean":"2023-11-28T11:17:50.200166+0000","last_became_active":"2023-11-28T11:17:49.216225+0000","last_became_peered":"2023-11-28T11:17:49.216225+0000","last_unstale":"2023-11-28T11:17:50.200166+0000","last_undegraded":"2023-11-28T11:17:50.200166+0000","last_fullsized":"2023-11-28T11:17:50.200166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:22.361832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970478+0000","last_change":"2023-11-28T11:17:49.219093+0000","last_active":"2023-11-28T11:17:50.970478+0000","last_peered":"2023-11-28T11:17:50.970478+0000","last_clean":"2023-11-28T11:17:50.970478+0000","last_became_active":"2023-11-28T11:17:49.218818+0000","last_became_peered":"2023-11-28T11:17:49.218818+0000","last_unstale":"2023-11-28T11:17:50.970478+0000","last_undegraded":"2023-11-28T11:17:50.970478+0000","last_fullsized":"2023-11-28T11:17:50.970478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198448+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.198448+0000","last_peered":"2023-11-28T11:17:50.198448+0000","last_clean":"2023-11-28T11:17:50.198448+0000","last_became_active":"2023-11-28T11:17:49.215424+0000","last_became_peered":"2023-11-28T11:17:49.215424+0000","last_unstale":"2023-11-28T11:17:50.198448+0000","last_undegraded":"2023-11-28T11:17:50.198448+0000","last_fullsized":"2023-11-28T11:17:50.198448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13188,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358652,"statfs":{"total":96636764160,"available":96623259648,"internally_reserved":0,"allocated":847872,"data_stored":674912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12068,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359772,"statfs":{"total":96636764160,"available":96624406528,"internally_reserved":0,"allocated":225280,"data_stored":82780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":81816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13160,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358680,"statfs":{"total":96636764160,"available":96623288320,"internally_reserved":0,"allocated":819200,"data_stored":672695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12004,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359836,"statfs":{"total":96636764160,"available":96624472064,"internally_reserved":0,"allocated":225280,"data_stored":82780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0510000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12136,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359704,"statfs":{"total":96636764160,"available":96624336896,"internally_reserved":0,"allocated":229376,"data_stored":82870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12152,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359688,"statfs":{"total":96636764160,"available":96624320512,"internally_reserved":0,"allocated":245760,"data_stored":84384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12076,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359764,"statfs":{"total":96636764160,"available":96624398336,"internally_reserved":0,"allocated":233472,"data_stored":83416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.492}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:17:59.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:18:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.102+0000 7fbe2b9ff700 1 Processor -- start 2023-11-28T11:18:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.102+0000 7fbe2b9ff700 1 -- start start 2023-11-28T11:18:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.103+0000 7fbe2b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 0x7fbe240b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.103+0000 7fbe2b9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe240cd7c0 0x7fbe240cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.103+0000 7fbe2b9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe240ce160 con 0x7fbe24161340 2023-11-28T11:18:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.103+0000 7fbe2b9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe240b5800 con 0x7fbe240cd7c0 2023-11-28T11:18:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.103+0000 7fbe2b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240d1950 con 0x7fbe2415c9b0 2023-11-28T11:18:00.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe240cd7c0 0x7fbe240cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe2979b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 0x7fbe240b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe28f9a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbe24161340 0x7fbe240d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:43680/0 (socket says 172.21.15.53:43680) 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe240cd7c0 0x7fbe240cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:38294/0 (socket says 172.21.15.53:38294) 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe28f9a700 1 -- 172.21.15.53:0/3414819175 learned_addr learned my addr 172.21.15.53:0/3414819175 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 -- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 msgr2=0x7fbe240b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 0x7fbe240b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 -- 172.21.15.53:0/3414819175 >> v1:172.21.15.157:6789/0 conn(0x7fbe24161340 legacy=0x7fbe240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe29f9c700 1 -- 172.21.15.53:0/3414819175 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240d24f0 con 0x7fbe240cd7c0 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.104+0000 7fbe2979b700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 0x7fbe240b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe29f9c700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe240cd7c0 0x7fbe240cdc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009d90 tx=0x7fbe1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bcc073dd0ebe66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe1bfff700 1 -- 172.21.15.53:0/3414819175 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00e070 con 0x7fbe240cd7c0 2023-11-28T11:18:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe1bfff700 1 -- 172.21.15.53:0/3414819175 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c00bb20 con 0x7fbe240cd7c0 2023-11-28T11:18:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe1bfff700 1 -- 172.21.15.53:0/3414819175 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c012430 con 0x7fbe240cd7c0 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe240cd7c0 msgr2=0x7fbe240cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe240cd7c0 0x7fbe240cdc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009d90 tx=0x7fbe1c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3414819175 shutdown_connections 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.105+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe240cd7c0 0x7fbe240cdc20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.106+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3414819175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe2415c9b0 0x7fbe240b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.106+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3414819175 >> 172.21.15.53:0/3414819175 conn(0x7fbe24072600 msgr2=0x7fbe240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.106+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3414819175 shutdown_connections 2023-11-28T11:18:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.106+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3414819175 wait complete. 2023-11-28T11:18:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.107+0000 7fbe2b9ff700 1 Processor -- start 2023-11-28T11:18:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.108+0000 7fbe2b9ff700 1 -- start start 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.108+0000 7fbe2b9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2b9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2415c9b0 0x7fbe2415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe24161340 0x7fbe2415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2b9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240c6630 con 0x7fbe240cd7c0 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2b9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe241e6660 con 0x7fbe2415c9b0 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe241e6940 con 0x7fbe24161340 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40282/0 (socket says 172.21.15.53:40282) 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 -- 172.21.15.53:0/3547630137 learned_addr learned my addr 172.21.15.53:0/3547630137 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 -- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe24161340 msgr2=0x7fbe2415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe24161340 0x7fbe2415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 -- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2415c9b0 msgr2=0x7fbe2415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2415c9b0 0x7fbe2415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 -- 172.21.15.53:0/3547630137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200094a0 con 0x7fbe240cd7c0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.109+0000 7fbe2979b700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ebe0 tx=0x7fbe20009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.110+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000c8b0 con 0x7fbe240cd7c0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.110+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000ca10 con 0x7fbe240cd7c0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.110+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000cda0 con 0x7fbe240cd7c0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.110+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c009a70 con 0x7fbe240cd7c0 2023-11-28T11:18:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.110+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe240d1300 con 0x7fbe240cd7c0 2023-11-28T11:18:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.111+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe240b5520 con 0x7fbe240cd7c0 2023-11-28T11:18:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.112+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe20019030 con 0x7fbe240cd7c0 2023-11-28T11:18:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.113+0000 7fbe19ffb700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 0x7fbe0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.113+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe20085d60 con 0x7fbe240cd7c0 2023-11-28T11:18:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.113+0000 7fbe28f9a700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 0x7fbe0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.114+0000 7fbe28f9a700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 0x7fbe0806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe24154420 tx=0x7fbe14007ad0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.117+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe200524d0 con 0x7fbe240cd7c0 2023-11-28T11:18:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.315+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe240d15e0 con 0x7fbe080682f0 2023-11-28T11:18:00.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.324+0000 7fbe19ffb700 1 -- 172.21.15.53:0/3547630137 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286613 (secure 0 0 0) 0x7fbe240d15e0 con 0x7fbe080682f0 2023-11-28T11:18:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 msgr2=0x7fbe0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 0x7fbe0806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe24154420 tx=0x7fbe14007ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 msgr2=0x7fbe240c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ebe0 tx=0x7fbe20009e20 comp rx=0 tx=0).stop 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 shutdown_connections 2023-11-28T11:18:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe080682f0 0x7fbe0806a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe24161340 0x7fbe2415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe240cd7c0 0x7fbe240c60f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 --2- 172.21.15.53:0/3547630137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2415c9b0 0x7fbe2415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.328+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 >> 172.21.15.53:0/3547630137 conn(0x7fbe24072600 msgr2=0x7fbe2406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.329+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 shutdown_connections 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.329+0000 7fbe2b9ff700 1 -- 172.21.15.53:0/3547630137 wait complete. 2023-11-28T11:18:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:18:00.348 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T11:17:58.933572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":98912,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":95872,"kb_avail":754875808,"statfs":{"total":773094113280,"available":772992827392,"internally_reserved":0,"allocated":3047424,"data_stored":1845653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":98172928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.741265"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964019+0000","last_change":"2023-11-28T11:17:49.213974+0000","last_active":"2023-11-28T11:17:50.964019+0000","last_peered":"2023-11-28T11:17:50.964019+0000","last_clean":"2023-11-28T11:17:50.964019+0000","last_became_active":"2023-11-28T11:17:49.213506+0000","last_became_peered":"2023-11-28T11:17:49.213506+0000","last_unstale":"2023-11-28T11:17:50.964019+0000","last_undegraded":"2023-11-28T11:17:50.964019+0000","last_fullsized":"2023-11-28T11:17:50.964019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199622+0000","last_change":"2023-11-28T11:17:49.228867+0000","last_active":"2023-11-28T11:17:50.199622+0000","last_peered":"2023-11-28T11:17:50.199622+0000","last_clean":"2023-11-28T11:17:50.199622+0000","last_became_active":"2023-11-28T11:17:49.228581+0000","last_became_peered":"2023-11-28T11:17:49.228581+0000","last_unstale":"2023-11-28T11:17:50.199622+0000","last_undegraded":"2023-11-28T11:17:50.199622+0000","last_fullsized":"2023-11-28T11:17:50.199622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:10.291664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200594+0000","last_change":"2023-11-28T11:17:49.223726+0000","last_active":"2023-11-28T11:17:50.200594+0000","last_peered":"2023-11-28T11:17:50.200594+0000","last_clean":"2023-11-28T11:17:50.200594+0000","last_became_active":"2023-11-28T11:17:49.222194+0000","last_became_peered":"2023-11-28T11:17:49.222194+0000","last_unstale":"2023-11-28T11:17:50.200594+0000","last_undegraded":"2023-11-28T11:17:50.200594+0000","last_fullsized":"2023-11-28T11:17:50.200594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:23.103321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967010+0000","last_change":"2023-11-28T11:17:49.214435+0000","last_active":"2023-11-28T11:17:50.967010+0000","last_peered":"2023-11-28T11:17:50.967010+0000","last_clean":"2023-11-28T11:17:50.967010+0000","last_became_active":"2023-11-28T11:17:49.214093+0000","last_became_peered":"2023-11-28T11:17:49.214093+0000","last_unstale":"2023-11-28T11:17:50.967010+0000","last_undegraded":"2023-11-28T11:17:50.967010+0000","last_fullsized":"2023-11-28T11:17:50.967010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:04.720586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972744+0000","last_change":"2023-11-28T11:17:49.217415+0000","last_active":"2023-11-28T11:17:50.972744+0000","last_peered":"2023-11-28T11:17:50.972744+0000","last_clean":"2023-11-28T11:17:50.972744+0000","last_became_active":"2023-11-28T11:17:49.217134+0000","last_became_peered":"2023-11-28T11:17:49.217134+0000","last_unstale":"2023-11-28T11:17:50.972744+0000","last_undegraded":"2023-11-28T11:17:50.972744+0000","last_fullsized":"2023-11-28T11:17:50.972744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200106+0000","last_change":"2023-11-28T11:17:49.208771+0000","last_active":"2023-11-28T11:17:50.200106+0000","last_peered":"2023-11-28T11:17:50.200106+0000","last_clean":"2023-11-28T11:17:50.200106+0000","last_became_active":"2023-11-28T11:17:49.208484+0000","last_became_peered":"2023-11-28T11:17:49.208484+0000","last_unstale":"2023-11-28T11:17:50.200106+0000","last_undegraded":"2023-11-28T11:17:50.200106+0000","last_fullsized":"2023-11-28T11:17:50.200106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:32.183119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965277+0000","last_change":"2023-11-28T11:17:49.220560+0000","last_active":"2023-11-28T11:17:50.965277+0000","last_peered":"2023-11-28T11:17:50.965277+0000","last_clean":"2023-11-28T11:17:50.965277+0000","last_became_active":"2023-11-28T11:17:49.220013+0000","last_became_peered":"2023-11-28T11:17:49.220013+0000","last_unstale":"2023-11-28T11:17:50.965277+0000","last_undegraded":"2023-11-28T11:17:50.965277+0000","last_fullsized":"2023-11-28T11:17:50.965277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199285+0000","last_change":"2023-11-28T11:17:49.207080+0000","last_active":"2023-11-28T11:17:50.199285+0000","last_peered":"2023-11-28T11:17:50.199285+0000","last_clean":"2023-11-28T11:17:50.199285+0000","last_became_active":"2023-11-28T11:17:49.206569+0000","last_became_peered":"2023-11-28T11:17:49.206569+0000","last_unstale":"2023-11-28T11:17:50.199285+0000","last_undegraded":"2023-11-28T11:17:50.199285+0000","last_fullsized":"2023-11-28T11:17:50.199285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:19.859853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972845+0000","last_change":"2023-11-28T11:17:49.214151+0000","last_active":"2023-11-28T11:17:50.972845+0000","last_peered":"2023-11-28T11:17:50.972845+0000","last_clean":"2023-11-28T11:17:50.972845+0000","last_became_active":"2023-11-28T11:17:49.213623+0000","last_became_peered":"2023-11-28T11:17:49.213623+0000","last_unstale":"2023-11-28T11:17:50.972845+0000","last_undegraded":"2023-11-28T11:17:50.972845+0000","last_fullsized":"2023-11-28T11:17:50.972845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199866+0000","last_change":"2023-11-28T11:17:49.223078+0000","last_active":"2023-11-28T11:17:50.199866+0000","last_peered":"2023-11-28T11:17:50.199866+0000","last_clean":"2023-11-28T11:17:50.199866+0000","last_became_active":"2023-11-28T11:17:49.221852+0000","last_became_peered":"2023-11-28T11:17:49.221852+0000","last_unstale":"2023-11-28T11:17:50.199866+0000","last_undegraded":"2023-11-28T11:17:50.199866+0000","last_fullsized":"2023-11-28T11:17:50.199866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:47.568223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200736+0000","last_change":"2023-11-28T11:17:49.223486+0000","last_active":"2023-11-28T11:17:50.200736+0000","last_peered":"2023-11-28T11:17:50.200736+0000","last_clean":"2023-11-28T11:17:50.200736+0000","last_became_active":"2023-11-28T11:17:49.221471+0000","last_became_peered":"2023-11-28T11:17:49.221471+0000","last_unstale":"2023-11-28T11:17:50.200736+0000","last_undegraded":"2023-11-28T11:17:50.200736+0000","last_fullsized":"2023-11-28T11:17:50.200736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.963990+0000","last_change":"2023-11-28T11:17:49.208546+0000","last_active":"2023-11-28T11:17:50.963990+0000","last_peered":"2023-11-28T11:17:50.963990+0000","last_clean":"2023-11-28T11:17:50.963990+0000","last_became_active":"2023-11-28T11:17:49.208144+0000","last_became_peered":"2023-11-28T11:17:49.208144+0000","last_unstale":"2023-11-28T11:17:50.963990+0000","last_undegraded":"2023-11-28T11:17:50.963990+0000","last_fullsized":"2023-11-28T11:17:50.963990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.963973+0000","last_change":"2023-11-28T11:17:49.210677+0000","last_active":"2023-11-28T11:17:50.963973+0000","last_peered":"2023-11-28T11:17:50.963973+0000","last_clean":"2023-11-28T11:17:50.963973+0000","last_became_active":"2023-11-28T11:17:49.210425+0000","last_became_peered":"2023-11-28T11:17:49.210425+0000","last_unstale":"2023-11-28T11:17:50.963973+0000","last_undegraded":"2023-11-28T11:17:50.963973+0000","last_fullsized":"2023-11-28T11:17:50.963973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199673+0000","last_change":"2023-11-28T11:17:49.207731+0000","last_active":"2023-11-28T11:17:50.199673+0000","last_peered":"2023-11-28T11:17:50.199673+0000","last_clean":"2023-11-28T11:17:50.199673+0000","last_became_active":"2023-11-28T11:17:49.207411+0000","last_became_peered":"2023-11-28T11:17:49.207411+0000","last_unstale":"2023-11-28T11:17:50.199673+0000","last_undegraded":"2023-11-28T11:17:50.199673+0000","last_fullsized":"2023-11-28T11:17:50.199673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:57.387438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965314+0000","last_change":"2023-11-28T11:17:49.220742+0000","last_active":"2023-11-28T11:17:50.965314+0000","last_peered":"2023-11-28T11:17:50.965314+0000","last_clean":"2023-11-28T11:17:50.965314+0000","last_became_active":"2023-11-28T11:17:49.220406+0000","last_became_peered":"2023-11-28T11:17:49.220406+0000","last_unstale":"2023-11-28T11:17:50.965314+0000","last_undegraded":"2023-11-28T11:17:50.965314+0000","last_fullsized":"2023-11-28T11:17:50.965314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199468+0000","last_change":"2023-11-28T11:17:49.205686+0000","last_active":"2023-11-28T11:17:50.199468+0000","last_peered":"2023-11-28T11:17:50.199468+0000","last_clean":"2023-11-28T11:17:50.199468+0000","last_became_active":"2023-11-28T11:17:49.205310+0000","last_became_peered":"2023-11-28T11:17:49.205310+0000","last_unstale":"2023-11-28T11:17:50.199468+0000","last_undegraded":"2023-11-28T11:17:50.199468+0000","last_fullsized":"2023-11-28T11:17:50.199468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:23.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972853+0000","last_change":"2023-11-28T11:17:49.213934+0000","last_active":"2023-11-28T11:17:50.972853+0000","last_peered":"2023-11-28T11:17:50.972853+0000","last_clean":"2023-11-28T11:17:50.972853+0000","last_became_active":"2023-11-28T11:17:49.213324+0000","last_became_peered":"2023-11-28T11:17:49.213324+0000","last_unstale":"2023-11-28T11:17:50.972853+0000","last_undegraded":"2023-11-28T11:17:50.972853+0000","last_fullsized":"2023-11-28T11:17:50.972853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200680+0000","last_change":"2023-11-28T11:17:49.207509+0000","last_active":"2023-11-28T11:17:50.200680+0000","last_peered":"2023-11-28T11:17:50.200680+0000","last_clean":"2023-11-28T11:17:50.200680+0000","last_became_active":"2023-11-28T11:17:49.206337+0000","last_became_peered":"2023-11-28T11:17:49.206337+0000","last_unstale":"2023-11-28T11:17:50.200680+0000","last_undegraded":"2023-11-28T11:17:50.200680+0000","last_fullsized":"2023-11-28T11:17:50.200680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964010+0000","last_change":"2023-11-28T11:17:49.213657+0000","last_active":"2023-11-28T11:17:50.964010+0000","last_peered":"2023-11-28T11:17:50.964010+0000","last_clean":"2023-11-28T11:17:50.964010+0000","last_became_active":"2023-11-28T11:17:49.213294+0000","last_became_peered":"2023-11-28T11:17:49.213294+0000","last_unstale":"2023-11-28T11:17:50.964010+0000","last_undegraded":"2023-11-28T11:17:50.964010+0000","last_fullsized":"2023-11-28T11:17:50.964010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:50.362636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200525+0000","last_change":"2023-11-28T11:17:49.211522+0000","last_active":"2023-11-28T11:17:50.200525+0000","last_peered":"2023-11-28T11:17:50.200525+0000","last_clean":"2023-11-28T11:17:50.200525+0000","last_became_active":"2023-11-28T11:17:49.211218+0000","last_became_peered":"2023-11-28T11:17:49.211218+0000","last_unstale":"2023-11-28T11:17:50.200525+0000","last_undegraded":"2023-11-28T11:17:50.200525+0000","last_fullsized":"2023-11-28T11:17:50.200525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:54.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965353+0000","last_change":"2023-11-28T11:17:49.223624+0000","last_active":"2023-11-28T11:17:50.965353+0000","last_peered":"2023-11-28T11:17:50.965353+0000","last_clean":"2023-11-28T11:17:50.965353+0000","last_became_active":"2023-11-28T11:17:49.222927+0000","last_became_peered":"2023-11-28T11:17:49.222927+0000","last_unstale":"2023-11-28T11:17:50.965353+0000","last_undegraded":"2023-11-28T11:17:50.965353+0000","last_fullsized":"2023-11-28T11:17:50.965353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972725+0000","last_change":"2023-11-28T11:17:49.225717+0000","last_active":"2023-11-28T11:17:50.972725+0000","last_peered":"2023-11-28T11:17:50.972725+0000","last_clean":"2023-11-28T11:17:50.972725+0000","last_became_active":"2023-11-28T11:17:49.225457+0000","last_became_peered":"2023-11-28T11:17:49.225457+0000","last_unstale":"2023-11-28T11:17:50.972725+0000","last_undegraded":"2023-11-28T11:17:50.972725+0000","last_fullsized":"2023-11-28T11:17:50.972725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970424+0000","last_change":"2023-11-28T11:17:49.214153+0000","last_active":"2023-11-28T11:17:50.970424+0000","last_peered":"2023-11-28T11:17:50.970424+0000","last_clean":"2023-11-28T11:17:50.970424+0000","last_became_active":"2023-11-28T11:17:49.213231+0000","last_became_peered":"2023-11-28T11:17:49.213231+0000","last_unstale":"2023-11-28T11:17:50.970424+0000","last_undegraded":"2023-11-28T11:17:50.970424+0000","last_fullsized":"2023-11-28T11:17:50.970424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965242+0000","last_change":"2023-11-28T11:17:49.219658+0000","last_active":"2023-11-28T11:17:50.965242+0000","last_peered":"2023-11-28T11:17:50.965242+0000","last_clean":"2023-11-28T11:17:50.965242+0000","last_became_active":"2023-11-28T11:17:49.219357+0000","last_became_peered":"2023-11-28T11:17:49.219357+0000","last_unstale":"2023-11-28T11:17:50.965242+0000","last_undegraded":"2023-11-28T11:17:50.965242+0000","last_fullsized":"2023-11-28T11:17:50.965242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198788+0000","last_change":"2023-11-28T11:17:49.224287+0000","last_active":"2023-11-28T11:17:50.198788+0000","last_peered":"2023-11-28T11:17:50.198788+0000","last_clean":"2023-11-28T11:17:50.198788+0000","last_became_active":"2023-11-28T11:17:49.223202+0000","last_became_peered":"2023-11-28T11:17:49.223202+0000","last_unstale":"2023-11-28T11:17:50.198788+0000","last_undegraded":"2023-11-28T11:17:50.198788+0000","last_fullsized":"2023-11-28T11:17:50.198788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198664+0000","last_change":"2023-11-28T11:17:49.223865+0000","last_active":"2023-11-28T11:17:50.198664+0000","last_peered":"2023-11-28T11:17:50.198664+0000","last_clean":"2023-11-28T11:17:50.198664+0000","last_became_active":"2023-11-28T11:17:49.223609+0000","last_became_peered":"2023-11-28T11:17:49.223609+0000","last_unstale":"2023-11-28T11:17:50.198664+0000","last_undegraded":"2023-11-28T11:17:50.198664+0000","last_fullsized":"2023-11-28T11:17:50.198664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199744+0000","last_change":"2023-11-28T11:17:49.223914+0000","last_active":"2023-11-28T11:17:50.199744+0000","last_peered":"2023-11-28T11:17:50.199744+0000","last_clean":"2023-11-28T11:17:50.199744+0000","last_became_active":"2023-11-28T11:17:49.222584+0000","last_became_peered":"2023-11-28T11:17:49.222584+0000","last_unstale":"2023-11-28T11:17:50.199744+0000","last_undegraded":"2023-11-28T11:17:50.199744+0000","last_fullsized":"2023-11-28T11:17:50.199744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:52.447012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200227+0000","last_change":"2023-11-28T11:17:48.699811+0000","last_active":"2023-11-28T11:17:50.200227+0000","last_peered":"2023-11-28T11:17:50.200227+0000","last_clean":"2023-11-28T11:17:50.200227+0000","last_became_active":"2023-11-28T11:17:48.699476+0000","last_became_peered":"2023-11-28T11:17:48.699476+0000","last_unstale":"2023-11-28T11:17:50.200227+0000","last_undegraded":"2023-11-28T11:17:50.200227+0000","last_fullsized":"2023-11-28T11:17:50.200227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970498+0000","last_change":"2023-11-28T11:17:49.209310+0000","last_active":"2023-11-28T11:17:50.970498+0000","last_peered":"2023-11-28T11:17:50.970498+0000","last_clean":"2023-11-28T11:17:50.970498+0000","last_became_active":"2023-11-28T11:17:49.209040+0000","last_became_peered":"2023-11-28T11:17:49.209040+0000","last_unstale":"2023-11-28T11:17:50.970498+0000","last_undegraded":"2023-11-28T11:17:50.970498+0000","last_fullsized":"2023-11-28T11:17:50.970498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198161+0000","last_change":"2023-11-28T11:17:48.685181+0000","last_active":"2023-11-28T11:17:50.198161+0000","last_peered":"2023-11-28T11:17:50.198161+0000","last_clean":"2023-11-28T11:17:50.198161+0000","last_became_active":"2023-11-28T11:17:48.684843+0000","last_became_peered":"2023-11-28T11:17:48.684843+0000","last_unstale":"2023-11-28T11:17:50.198161+0000","last_undegraded":"2023-11-28T11:17:50.198161+0000","last_fullsized":"2023-11-28T11:17:50.198161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972795+0000","last_change":"2023-11-28T11:17:49.224963+0000","last_active":"2023-11-28T11:17:50.972795+0000","last_peered":"2023-11-28T11:17:50.972795+0000","last_clean":"2023-11-28T11:17:50.972795+0000","last_became_active":"2023-11-28T11:17:49.224679+0000","last_became_peered":"2023-11-28T11:17:49.224679+0000","last_unstale":"2023-11-28T11:17:50.972795+0000","last_undegraded":"2023-11-28T11:17:50.972795+0000","last_fullsized":"2023-11-28T11:17:50.972795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970396+0000","last_change":"2023-11-28T11:17:48.690687+0000","last_active":"2023-11-28T11:17:50.970396+0000","last_peered":"2023-11-28T11:17:50.970396+0000","last_clean":"2023-11-28T11:17:50.970396+0000","last_became_active":"2023-11-28T11:17:48.690509+0000","last_became_peered":"2023-11-28T11:17:48.690509+0000","last_unstale":"2023-11-28T11:17:50.970396+0000","last_undegraded":"2023-11-28T11:17:50.970396+0000","last_fullsized":"2023-11-28T11:17:50.970396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967085+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.967085+0000","last_peered":"2023-11-28T11:17:50.967085+0000","last_clean":"2023-11-28T11:17:50.967085+0000","last_became_active":"2023-11-28T11:17:49.215446+0000","last_became_peered":"2023-11-28T11:17:49.215446+0000","last_unstale":"2023-11-28T11:17:50.967085+0000","last_undegraded":"2023-11-28T11:17:50.967085+0000","last_fullsized":"2023-11-28T11:17:50.967085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:27:06.007692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972862+0000","last_change":"2023-11-28T11:17:48.689522+0000","last_active":"2023-11-28T11:17:50.972862+0000","last_peered":"2023-11-28T11:17:50.972862+0000","last_clean":"2023-11-28T11:17:50.972862+0000","last_became_active":"2023-11-28T11:17:48.689211+0000","last_became_peered":"2023-11-28T11:17:48.689211+0000","last_unstale":"2023-11-28T11:17:50.972862+0000","last_undegraded":"2023-11-28T11:17:50.972862+0000","last_fullsized":"2023-11-28T11:17:50.972862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970382+0000","last_change":"2023-11-28T11:17:49.222548+0000","last_active":"2023-11-28T11:17:50.970382+0000","last_peered":"2023-11-28T11:17:50.970382+0000","last_clean":"2023-11-28T11:17:50.970382+0000","last_became_active":"2023-11-28T11:17:49.222341+0000","last_became_peered":"2023-11-28T11:17:49.222341+0000","last_unstale":"2023-11-28T11:17:50.970382+0000","last_undegraded":"2023-11-28T11:17:50.970382+0000","last_fullsized":"2023-11-28T11:17:50.970382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199728+0000","last_change":"2023-11-28T11:17:48.696385+0000","last_active":"2023-11-28T11:17:50.199728+0000","last_peered":"2023-11-28T11:17:50.199728+0000","last_clean":"2023-11-28T11:17:50.199728+0000","last_became_active":"2023-11-28T11:17:48.696028+0000","last_became_peered":"2023-11-28T11:17:48.696028+0000","last_unstale":"2023-11-28T11:17:50.199728+0000","last_undegraded":"2023-11-28T11:17:50.199728+0000","last_fullsized":"2023-11-28T11:17:50.199728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:49.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970434+0000","last_change":"2023-11-28T11:17:49.210958+0000","last_active":"2023-11-28T11:17:50.970434+0000","last_peered":"2023-11-28T11:17:50.970434+0000","last_clean":"2023-11-28T11:17:50.970434+0000","last_became_active":"2023-11-28T11:17:49.210651+0000","last_became_peered":"2023-11-28T11:17:49.210651+0000","last_unstale":"2023-11-28T11:17:50.970434+0000","last_undegraded":"2023-11-28T11:17:50.970434+0000","last_fullsized":"2023-11-28T11:17:50.970434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965376+0000","last_change":"2023-11-28T11:17:48.697605+0000","last_active":"2023-11-28T11:17:50.965376+0000","last_peered":"2023-11-28T11:17:50.965376+0000","last_clean":"2023-11-28T11:17:50.965376+0000","last_became_active":"2023-11-28T11:17:48.696308+0000","last_became_peered":"2023-11-28T11:17:48.696308+0000","last_unstale":"2023-11-28T11:17:50.965376+0000","last_undegraded":"2023-11-28T11:17:50.965376+0000","last_fullsized":"2023-11-28T11:17:50.965376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:46.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198244+0000","last_change":"2023-11-28T11:17:49.213923+0000","last_active":"2023-11-28T11:17:50.198244+0000","last_peered":"2023-11-28T11:17:50.198244+0000","last_clean":"2023-11-28T11:17:50.198244+0000","last_became_active":"2023-11-28T11:17:49.213693+0000","last_became_peered":"2023-11-28T11:17:49.213693+0000","last_unstale":"2023-11-28T11:17:50.198244+0000","last_undegraded":"2023-11-28T11:17:50.198244+0000","last_fullsized":"2023-11-28T11:17:50.198244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:12.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200624+0000","last_change":"2023-11-28T11:17:48.693861+0000","last_active":"2023-11-28T11:17:50.200624+0000","last_peered":"2023-11-28T11:17:50.200624+0000","last_clean":"2023-11-28T11:17:50.200624+0000","last_became_active":"2023-11-28T11:17:48.693491+0000","last_became_peered":"2023-11-28T11:17:48.693491+0000","last_unstale":"2023-11-28T11:17:50.200624+0000","last_undegraded":"2023-11-28T11:17:50.200624+0000","last_fullsized":"2023-11-28T11:17:50.200624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:56.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200248+0000","last_change":"2023-11-28T11:17:49.213236+0000","last_active":"2023-11-28T11:17:50.200248+0000","last_peered":"2023-11-28T11:17:50.200248+0000","last_clean":"2023-11-28T11:17:50.200248+0000","last_became_active":"2023-11-28T11:17:49.211752+0000","last_became_peered":"2023-11-28T11:17:49.211752+0000","last_unstale":"2023-11-28T11:17:50.200248+0000","last_undegraded":"2023-11-28T11:17:50.200248+0000","last_fullsized":"2023-11-28T11:17:50.200248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:40.807651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965333+0000","last_change":"2023-11-28T11:17:48.702529+0000","last_active":"2023-11-28T11:17:50.965333+0000","last_peered":"2023-11-28T11:17:50.965333+0000","last_clean":"2023-11-28T11:17:50.965333+0000","last_became_active":"2023-11-28T11:17:48.702137+0000","last_became_peered":"2023-11-28T11:17:48.702137+0000","last_unstale":"2023-11-28T11:17:50.965333+0000","last_undegraded":"2023-11-28T11:17:50.965333+0000","last_fullsized":"2023-11-28T11:17:50.965333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:49.562684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970489+0000","last_change":"2023-11-28T11:17:49.223453+0000","last_active":"2023-11-28T11:17:50.970489+0000","last_peered":"2023-11-28T11:17:50.970489+0000","last_clean":"2023-11-28T11:17:50.970489+0000","last_became_active":"2023-11-28T11:17:49.222771+0000","last_became_peered":"2023-11-28T11:17:49.222771+0000","last_unstale":"2023-11-28T11:17:50.970489+0000","last_undegraded":"2023-11-28T11:17:50.970489+0000","last_fullsized":"2023-11-28T11:17:50.970489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972714+0000","last_change":"2023-11-28T11:17:48.689983+0000","last_active":"2023-11-28T11:17:50.972714+0000","last_peered":"2023-11-28T11:17:50.972714+0000","last_clean":"2023-11-28T11:17:50.972714+0000","last_became_active":"2023-11-28T11:17:48.688579+0000","last_became_peered":"2023-11-28T11:17:48.688579+0000","last_unstale":"2023-11-28T11:17:50.972714+0000","last_undegraded":"2023-11-28T11:17:50.972714+0000","last_fullsized":"2023-11-28T11:17:50.972714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199828+0000","last_change":"2023-11-28T11:17:49.219334+0000","last_active":"2023-11-28T11:17:50.199828+0000","last_peered":"2023-11-28T11:17:50.199828+0000","last_clean":"2023-11-28T11:17:50.199828+0000","last_became_active":"2023-11-28T11:17:49.218912+0000","last_became_peered":"2023-11-28T11:17:49.218912+0000","last_unstale":"2023-11-28T11:17:50.199828+0000","last_undegraded":"2023-11-28T11:17:50.199828+0000","last_fullsized":"2023-11-28T11:17:50.199828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:29.752626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965296+0000","last_change":"2023-11-28T11:17:48.702427+0000","last_active":"2023-11-28T11:17:50.965296+0000","last_peered":"2023-11-28T11:17:50.965296+0000","last_clean":"2023-11-28T11:17:50.965296+0000","last_became_active":"2023-11-28T11:17:48.702118+0000","last_became_peered":"2023-11-28T11:17:48.702118+0000","last_unstale":"2023-11-28T11:17:50.965296+0000","last_undegraded":"2023-11-28T11:17:50.965296+0000","last_fullsized":"2023-11-28T11:17:50.965296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:37.239541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972785+0000","last_change":"2023-11-28T11:17:49.209958+0000","last_active":"2023-11-28T11:17:50.972785+0000","last_peered":"2023-11-28T11:17:50.972785+0000","last_clean":"2023-11-28T11:17:50.972785+0000","last_became_active":"2023-11-28T11:17:49.209616+0000","last_became_peered":"2023-11-28T11:17:49.209616+0000","last_unstale":"2023-11-28T11:17:50.972785+0000","last_undegraded":"2023-11-28T11:17:50.972785+0000","last_fullsized":"2023-11-28T11:17:50.972785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:37.484246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966956+0000","last_change":"2023-11-28T11:17:48.696749+0000","last_active":"2023-11-28T11:17:50.966956+0000","last_peered":"2023-11-28T11:17:50.966956+0000","last_clean":"2023-11-28T11:17:50.966956+0000","last_became_active":"2023-11-28T11:17:48.695400+0000","last_became_peered":"2023-11-28T11:17:48.695400+0000","last_unstale":"2023-11-28T11:17:50.966956+0000","last_undegraded":"2023-11-28T11:17:50.966956+0000","last_fullsized":"2023-11-28T11:17:50.966956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970533+0000","last_change":"2023-11-28T11:17:49.209165+0000","last_active":"2023-11-28T11:17:50.970533+0000","last_peered":"2023-11-28T11:17:50.970533+0000","last_clean":"2023-11-28T11:17:50.970533+0000","last_became_active":"2023-11-28T11:17:49.208916+0000","last_became_peered":"2023-11-28T11:17:49.208916+0000","last_unstale":"2023-11-28T11:17:50.970533+0000","last_undegraded":"2023-11-28T11:17:50.970533+0000","last_fullsized":"2023-11-28T11:17:50.970533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965478+0000","last_change":"2023-11-28T11:17:48.700934+0000","last_active":"2023-11-28T11:17:50.965478+0000","last_peered":"2023-11-28T11:17:50.965478+0000","last_clean":"2023-11-28T11:17:50.965478+0000","last_became_active":"2023-11-28T11:17:48.700549+0000","last_became_peered":"2023-11-28T11:17:48.700549+0000","last_unstale":"2023-11-28T11:17:50.965478+0000","last_undegraded":"2023-11-28T11:17:50.965478+0000","last_fullsized":"2023-11-28T11:17:50.965478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:40.639240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199331+0000","last_change":"2023-11-28T11:17:49.218909+0000","last_active":"2023-11-28T11:17:50.199331+0000","last_peered":"2023-11-28T11:17:50.199331+0000","last_clean":"2023-11-28T11:17:50.199331+0000","last_became_active":"2023-11-28T11:17:49.218582+0000","last_became_peered":"2023-11-28T11:17:49.218582+0000","last_unstale":"2023-11-28T11:17:50.199331+0000","last_undegraded":"2023-11-28T11:17:50.199331+0000","last_fullsized":"2023-11-28T11:17:50.199331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:25.463820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965454+0000","last_change":"2023-11-28T11:17:48.698986+0000","last_active":"2023-11-28T11:17:50.965454+0000","last_peered":"2023-11-28T11:17:50.965454+0000","last_clean":"2023-11-28T11:17:50.965454+0000","last_became_active":"2023-11-28T11:17:48.698594+0000","last_became_peered":"2023-11-28T11:17:48.698594+0000","last_unstale":"2023-11-28T11:17:50.965454+0000","last_undegraded":"2023-11-28T11:17:50.965454+0000","last_fullsized":"2023-11-28T11:17:50.965454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:19.087105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199992+0000","last_change":"2023-11-28T11:17:49.213092+0000","last_active":"2023-11-28T11:17:50.199992+0000","last_peered":"2023-11-28T11:17:50.199992+0000","last_clean":"2023-11-28T11:17:50.199992+0000","last_became_active":"2023-11-28T11:17:49.211870+0000","last_became_peered":"2023-11-28T11:17:49.211870+0000","last_unstale":"2023-11-28T11:17:50.199992+0000","last_undegraded":"2023-11-28T11:17:50.199992+0000","last_fullsized":"2023-11-28T11:17:50.199992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:41.645456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.965432+0000","last_change":"2023-11-28T11:17:48.695292+0000","last_active":"2023-11-28T11:17:50.965432+0000","last_peered":"2023-11-28T11:17:50.965432+0000","last_clean":"2023-11-28T11:17:50.965432+0000","last_became_active":"2023-11-28T11:17:48.694985+0000","last_became_peered":"2023-11-28T11:17:48.694985+0000","last_unstale":"2023-11-28T11:17:50.965432+0000","last_undegraded":"2023-11-28T11:17:50.965432+0000","last_fullsized":"2023-11-28T11:17:50.965432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:56.077529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966999+0000","last_change":"2023-11-28T11:17:49.226629+0000","last_active":"2023-11-28T11:17:50.966999+0000","last_peered":"2023-11-28T11:17:50.966999+0000","last_clean":"2023-11-28T11:17:50.966999+0000","last_became_active":"2023-11-28T11:17:49.226347+0000","last_became_peered":"2023-11-28T11:17:49.226347+0000","last_unstale":"2023-11-28T11:17:50.966999+0000","last_undegraded":"2023-11-28T11:17:50.966999+0000","last_fullsized":"2023-11-28T11:17:50.966999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199618+0000","last_change":"2023-11-28T11:17:48.692881+0000","last_active":"2023-11-28T11:17:50.199618+0000","last_peered":"2023-11-28T11:17:50.199618+0000","last_clean":"2023-11-28T11:17:50.199618+0000","last_became_active":"2023-11-28T11:17:48.692426+0000","last_became_peered":"2023-11-28T11:17:48.692426+0000","last_unstale":"2023-11-28T11:17:50.199618+0000","last_undegraded":"2023-11-28T11:17:50.199618+0000","last_fullsized":"2023-11-28T11:17:50.199618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:23.247523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972735+0000","last_change":"2023-11-28T11:17:49.221185+0000","last_active":"2023-11-28T11:17:50.972735+0000","last_peered":"2023-11-28T11:17:50.972735+0000","last_clean":"2023-11-28T11:17:50.972735+0000","last_became_active":"2023-11-28T11:17:49.220927+0000","last_became_peered":"2023-11-28T11:17:49.220927+0000","last_unstale":"2023-11-28T11:17:50.972735+0000","last_undegraded":"2023-11-28T11:17:50.972735+0000","last_fullsized":"2023-11-28T11:17:50.972735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970455+0000","last_change":"2023-11-28T11:17:48.698335+0000","last_active":"2023-11-28T11:17:50.970455+0000","last_peered":"2023-11-28T11:17:50.970455+0000","last_clean":"2023-11-28T11:17:50.970455+0000","last_became_active":"2023-11-28T11:17:48.698032+0000","last_became_peered":"2023-11-28T11:17:48.698032+0000","last_unstale":"2023-11-28T11:17:50.970455+0000","last_undegraded":"2023-11-28T11:17:50.970455+0000","last_fullsized":"2023-11-28T11:17:50.970455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200609+0000","last_change":"2023-11-28T11:17:49.207155+0000","last_active":"2023-11-28T11:17:50.200609+0000","last_peered":"2023-11-28T11:17:50.200609+0000","last_clean":"2023-11-28T11:17:50.200609+0000","last_became_active":"2023-11-28T11:17:49.206800+0000","last_became_peered":"2023-11-28T11:17:49.206800+0000","last_unstale":"2023-11-28T11:17:50.200609+0000","last_undegraded":"2023-11-28T11:17:50.200609+0000","last_fullsized":"2023-11-28T11:17:50.200609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:31.904697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966985+0000","last_change":"2023-11-28T11:17:48.702363+0000","last_active":"2023-11-28T11:17:50.966985+0000","last_peered":"2023-11-28T11:17:50.966985+0000","last_clean":"2023-11-28T11:17:50.966985+0000","last_became_active":"2023-11-28T11:17:48.701778+0000","last_became_peered":"2023-11-28T11:17:48.701778+0000","last_unstale":"2023-11-28T11:17:50.966985+0000","last_undegraded":"2023-11-28T11:17:50.966985+0000","last_fullsized":"2023-11-28T11:17:50.966985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:53.049772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200788+0000","last_change":"2023-11-28T11:17:49.205845+0000","last_active":"2023-11-28T11:17:50.200788+0000","last_peered":"2023-11-28T11:17:50.200788+0000","last_clean":"2023-11-28T11:17:50.200788+0000","last_became_active":"2023-11-28T11:17:49.205302+0000","last_became_peered":"2023-11-28T11:17:49.205302+0000","last_unstale":"2023-11-28T11:17:50.200788+0000","last_undegraded":"2023-11-28T11:17:50.200788+0000","last_fullsized":"2023-11-28T11:17:50.200788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:24.286473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200199+0000","last_change":"2023-11-28T11:17:48.699968+0000","last_active":"2023-11-28T11:17:50.200199+0000","last_peered":"2023-11-28T11:17:50.200199+0000","last_clean":"2023-11-28T11:17:50.200199+0000","last_became_active":"2023-11-28T11:17:48.697897+0000","last_became_peered":"2023-11-28T11:17:48.697897+0000","last_unstale":"2023-11-28T11:17:50.200199+0000","last_undegraded":"2023-11-28T11:17:50.200199+0000","last_fullsized":"2023-11-28T11:17:50.200199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:10.154840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200758+0000","last_change":"2023-11-28T11:17:49.210939+0000","last_active":"2023-11-28T11:17:50.200758+0000","last_peered":"2023-11-28T11:17:50.200758+0000","last_clean":"2023-11-28T11:17:50.200758+0000","last_became_active":"2023-11-28T11:17:49.210585+0000","last_became_peered":"2023-11-28T11:17:49.210585+0000","last_unstale":"2023-11-28T11:17:50.200758+0000","last_undegraded":"2023-11-28T11:17:50.200758+0000","last_fullsized":"2023-11-28T11:17:50.200758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:17.930855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970354+0000","last_change":"2023-11-28T11:17:48.687630+0000","last_active":"2023-11-28T11:17:50.970354+0000","last_peered":"2023-11-28T11:17:50.970354+0000","last_clean":"2023-11-28T11:17:50.970354+0000","last_became_active":"2023-11-28T11:17:48.685494+0000","last_became_peered":"2023-11-28T11:17:48.685494+0000","last_unstale":"2023-11-28T11:17:50.970354+0000","last_undegraded":"2023-11-28T11:17:50.970354+0000","last_fullsized":"2023-11-28T11:17:50.970354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198866+0000","last_change":"2023-11-28T11:17:49.223631+0000","last_active":"2023-11-28T11:17:50.198866+0000","last_peered":"2023-11-28T11:17:50.198866+0000","last_clean":"2023-11-28T11:17:50.198866+0000","last_became_active":"2023-11-28T11:17:49.223321+0000","last_became_peered":"2023-11-28T11:17:49.223321+0000","last_unstale":"2023-11-28T11:17:50.198866+0000","last_undegraded":"2023-11-28T11:17:50.198866+0000","last_fullsized":"2023-11-28T11:17:50.198866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:38.110221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.195384+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:17:49.195384+0000","last_peered":"2023-11-28T11:17:49.195384+0000","last_clean":"2023-11-28T11:17:49.195384+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:17:49.195384+0000","last_undegraded":"2023-11-28T11:17:49.195384+0000","last_fullsized":"2023-11-28T11:17:49.195384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199900+0000","last_change":"2023-11-28T11:17:48.691483+0000","last_active":"2023-11-28T11:17:50.199900+0000","last_peered":"2023-11-28T11:17:50.199900+0000","last_clean":"2023-11-28T11:17:50.199900+0000","last_became_active":"2023-11-28T11:17:48.691194+0000","last_became_peered":"2023-11-28T11:17:48.691194+0000","last_unstale":"2023-11-28T11:17:50.199900+0000","last_undegraded":"2023-11-28T11:17:50.199900+0000","last_fullsized":"2023-11-28T11:17:50.199900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:43.938903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198294+0000","last_change":"2023-11-28T11:17:49.217104+0000","last_active":"2023-11-28T11:17:50.198294+0000","last_peered":"2023-11-28T11:17:50.198294+0000","last_clean":"2023-11-28T11:17:50.198294+0000","last_became_active":"2023-11-28T11:17:49.216883+0000","last_became_peered":"2023-11-28T11:17:49.216883+0000","last_unstale":"2023-11-28T11:17:50.198294+0000","last_undegraded":"2023-11-28T11:17:50.198294+0000","last_fullsized":"2023-11-28T11:17:50.198294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202595+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:17:49.202595+0000","last_peered":"2023-11-28T11:17:49.202595+0000","last_clean":"2023-11-28T11:17:49.202595+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:17:49.202595+0000","last_undegraded":"2023-11-28T11:17:49.202595+0000","last_fullsized":"2023-11-28T11:17:49.202595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200374+0000","last_change":"2023-11-28T11:17:48.699726+0000","last_active":"2023-11-28T11:17:50.200374+0000","last_peered":"2023-11-28T11:17:50.200374+0000","last_clean":"2023-11-28T11:17:50.200374+0000","last_became_active":"2023-11-28T11:17:48.699409+0000","last_became_peered":"2023-11-28T11:17:48.699409+0000","last_unstale":"2023-11-28T11:17:50.200374+0000","last_undegraded":"2023-11-28T11:17:50.200374+0000","last_fullsized":"2023-11-28T11:17:50.200374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198558+0000","last_change":"2023-11-28T11:17:49.223120+0000","last_active":"2023-11-28T11:17:50.198558+0000","last_peered":"2023-11-28T11:17:50.198558+0000","last_clean":"2023-11-28T11:17:50.198558+0000","last_became_active":"2023-11-28T11:17:49.217684+0000","last_became_peered":"2023-11-28T11:17:49.217684+0000","last_unstale":"2023-11-28T11:17:50.198558+0000","last_undegraded":"2023-11-28T11:17:50.198558+0000","last_fullsized":"2023-11-28T11:17:50.198558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:57.263922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199240+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:17:49.199240+0000","last_peered":"2023-11-28T11:17:49.199240+0000","last_clean":"2023-11-28T11:17:49.199240+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:17:49.199240+0000","last_undegraded":"2023-11-28T11:17:49.199240+0000","last_fullsized":"2023-11-28T11:17:49.199240+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198094+0000","last_change":"2023-11-28T11:17:48.695369+0000","last_active":"2023-11-28T11:17:50.198094+0000","last_peered":"2023-11-28T11:17:50.198094+0000","last_clean":"2023-11-28T11:17:50.198094+0000","last_became_active":"2023-11-28T11:17:48.695065+0000","last_became_peered":"2023-11-28T11:17:48.695065+0000","last_unstale":"2023-11-28T11:17:50.198094+0000","last_undegraded":"2023-11-28T11:17:50.198094+0000","last_fullsized":"2023-11-28T11:17:50.198094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198772+0000","last_change":"2023-11-28T11:17:49.217116+0000","last_active":"2023-11-28T11:17:50.198772+0000","last_peered":"2023-11-28T11:17:50.198772+0000","last_clean":"2023-11-28T11:17:50.198772+0000","last_became_active":"2023-11-28T11:17:49.216818+0000","last_became_peered":"2023-11-28T11:17:49.216818+0000","last_unstale":"2023-11-28T11:17:50.198772+0000","last_undegraded":"2023-11-28T11:17:50.198772+0000","last_fullsized":"2023-11-28T11:17:50.198772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.202500+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:17:49.202500+0000","last_peered":"2023-11-28T11:17:49.202500+0000","last_clean":"2023-11-28T11:17:49.202500+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:17:49.202500+0000","last_undegraded":"2023-11-28T11:17:49.202500+0000","last_fullsized":"2023-11-28T11:17:49.202500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.198393+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:17:49.198393+0000","last_peered":"2023-11-28T11:17:49.198393+0000","last_clean":"2023-11-28T11:17:49.198393+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:17:49.198393+0000","last_undegraded":"2023-11-28T11:17:49.198393+0000","last_fullsized":"2023-11-28T11:17:49.198393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970443+0000","last_change":"2023-11-28T11:17:48.692995+0000","last_active":"2023-11-28T11:17:50.970443+0000","last_peered":"2023-11-28T11:17:50.970443+0000","last_clean":"2023-11-28T11:17:50.970443+0000","last_became_active":"2023-11-28T11:17:48.692619+0000","last_became_peered":"2023-11-28T11:17:48.692619+0000","last_unstale":"2023-11-28T11:17:50.970443+0000","last_undegraded":"2023-11-28T11:17:50.970443+0000","last_fullsized":"2023-11-28T11:17:50.970443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.967024+0000","last_change":"2023-11-28T11:17:49.211592+0000","last_active":"2023-11-28T11:17:50.967024+0000","last_peered":"2023-11-28T11:17:50.967024+0000","last_clean":"2023-11-28T11:17:50.967024+0000","last_became_active":"2023-11-28T11:17:49.209606+0000","last_became_peered":"2023-11-28T11:17:49.209606+0000","last_unstale":"2023-11-28T11:17:50.967024+0000","last_undegraded":"2023-11-28T11:17:50.967024+0000","last_fullsized":"2023-11-28T11:17:50.967024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:47.444676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199814+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:17:49.199814+0000","last_peered":"2023-11-28T11:17:49.199814+0000","last_clean":"2023-11-28T11:17:49.199814+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:17:49.199814+0000","last_undegraded":"2023-11-28T11:17:49.199814+0000","last_fullsized":"2023-11-28T11:17:49.199814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972688+0000","last_change":"2023-11-28T11:17:48.696786+0000","last_active":"2023-11-28T11:17:50.972688+0000","last_peered":"2023-11-28T11:17:50.972688+0000","last_clean":"2023-11-28T11:17:50.972688+0000","last_became_active":"2023-11-28T11:17:48.696474+0000","last_became_peered":"2023-11-28T11:17:48.696474+0000","last_unstale":"2023-11-28T11:17:50.972688+0000","last_undegraded":"2023-11-28T11:17:50.972688+0000","last_fullsized":"2023-11-28T11:17:50.972688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972836+0000","last_change":"2023-11-28T11:17:49.219523+0000","last_active":"2023-11-28T11:17:50.972836+0000","last_peered":"2023-11-28T11:17:50.972836+0000","last_clean":"2023-11-28T11:17:50.972836+0000","last_became_active":"2023-11-28T11:17:49.219226+0000","last_became_peered":"2023-11-28T11:17:49.219226+0000","last_unstale":"2023-11-28T11:17:50.972836+0000","last_undegraded":"2023-11-28T11:17:50.972836+0000","last_fullsized":"2023-11-28T11:17:50.972836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.199998+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:17:49.199998+0000","last_peered":"2023-11-28T11:17:49.199998+0000","last_clean":"2023-11-28T11:17:49.199998+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:17:49.199998+0000","last_undegraded":"2023-11-28T11:17:49.199998+0000","last_fullsized":"2023-11-28T11:17:49.199998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198637+0000","last_change":"2023-11-28T11:17:48.696130+0000","last_active":"2023-11-28T11:17:50.198637+0000","last_peered":"2023-11-28T11:17:50.198637+0000","last_clean":"2023-11-28T11:17:50.198637+0000","last_became_active":"2023-11-28T11:17:48.695765+0000","last_became_peered":"2023-11-28T11:17:48.695765+0000","last_unstale":"2023-11-28T11:17:50.198637+0000","last_undegraded":"2023-11-28T11:17:50.198637+0000","last_fullsized":"2023-11-28T11:17:50.198637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:46.695657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972753+0000","last_change":"2023-11-28T11:17:49.219689+0000","last_active":"2023-11-28T11:17:50.972753+0000","last_peered":"2023-11-28T11:17:50.972753+0000","last_clean":"2023-11-28T11:17:50.972753+0000","last_became_active":"2023-11-28T11:17:49.219446+0000","last_became_peered":"2023-11-28T11:17:49.219446+0000","last_unstale":"2023-11-28T11:17:50.972753+0000","last_undegraded":"2023-11-28T11:17:50.972753+0000","last_fullsized":"2023-11-28T11:17:50.972753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:49.108057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970522+0000","last_change":"2023-11-28T11:17:48.694653+0000","last_active":"2023-11-28T11:17:50.970522+0000","last_peered":"2023-11-28T11:17:50.970522+0000","last_clean":"2023-11-28T11:17:50.970522+0000","last_became_active":"2023-11-28T11:17:48.694360+0000","last_became_peered":"2023-11-28T11:17:48.694360+0000","last_unstale":"2023-11-28T11:17:50.970522+0000","last_undegraded":"2023-11-28T11:17:50.970522+0000","last_fullsized":"2023-11-28T11:17:50.970522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:34.424868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.194295+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:17:49.194295+0000","last_peered":"2023-11-28T11:17:49.194295+0000","last_clean":"2023-11-28T11:17:49.194295+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:17:49.194295+0000","last_undegraded":"2023-11-28T11:17:49.194295+0000","last_fullsized":"2023-11-28T11:17:49.194295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970337+0000","last_change":"2023-11-28T11:17:49.219150+0000","last_active":"2023-11-28T11:17:50.970337+0000","last_peered":"2023-11-28T11:17:50.970337+0000","last_clean":"2023-11-28T11:17:50.970337+0000","last_became_active":"2023-11-28T11:17:49.218887+0000","last_became_peered":"2023-11-28T11:17:49.218887+0000","last_unstale":"2023-11-28T11:17:50.970337+0000","last_undegraded":"2023-11-28T11:17:50.970337+0000","last_fullsized":"2023-11-28T11:17:50.970337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:59.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:49.204699+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:17:49.204699+0000","last_peered":"2023-11-28T11:17:49.204699+0000","last_clean":"2023-11-28T11:17:49.204699+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:17:49.204699+0000","last_undegraded":"2023-11-28T11:17:49.204699+0000","last_fullsized":"2023-11-28T11:17:49.204699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:51.413066+0000","last_change":"2023-11-28T11:17:48.688812+0000","last_active":"2023-11-28T11:17:51.413066+0000","last_peered":"2023-11-28T11:17:51.413066+0000","last_clean":"2023-11-28T11:17:51.413066+0000","last_became_active":"2023-11-28T11:17:48.688551+0000","last_became_peered":"2023-11-28T11:17:48.688551+0000","last_unstale":"2023-11-28T11:17:51.413066+0000","last_undegraded":"2023-11-28T11:17:51.413066+0000","last_fullsized":"2023-11-28T11:17:51.413066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:34.062586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200103+0000","last_change":"2023-11-28T11:17:49.207524+0000","last_active":"2023-11-28T11:17:50.200103+0000","last_peered":"2023-11-28T11:17:50.200103+0000","last_clean":"2023-11-28T11:17:50.200103+0000","last_became_active":"2023-11-28T11:17:49.207175+0000","last_became_peered":"2023-11-28T11:17:49.207175+0000","last_unstale":"2023-11-28T11:17:50.200103+0000","last_undegraded":"2023-11-28T11:17:50.200103+0000","last_fullsized":"2023-11-28T11:17:50.200103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:42.481462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972774+0000","last_change":"2023-11-28T11:17:48.691892+0000","last_active":"2023-11-28T11:17:50.972774+0000","last_peered":"2023-11-28T11:17:50.972774+0000","last_clean":"2023-11-28T11:17:50.972774+0000","last_became_active":"2023-11-28T11:17:48.691619+0000","last_became_peered":"2023-11-28T11:17:48.691619+0000","last_unstale":"2023-11-28T11:17:50.972774+0000","last_undegraded":"2023-11-28T11:17:50.972774+0000","last_fullsized":"2023-11-28T11:17:50.972774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:57.955579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.964001+0000","last_change":"2023-11-28T11:17:49.230012+0000","last_active":"2023-11-28T11:17:50.964001+0000","last_peered":"2023-11-28T11:17:50.964001+0000","last_clean":"2023-11-28T11:17:50.964001+0000","last_became_active":"2023-11-28T11:17:49.229696+0000","last_became_peered":"2023-11-28T11:17:49.229696+0000","last_unstale":"2023-11-28T11:17:50.964001+0000","last_undegraded":"2023-11-28T11:17:50.964001+0000","last_fullsized":"2023-11-28T11:17:50.964001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199712+0000","last_change":"2023-11-28T11:17:48.699672+0000","last_active":"2023-11-28T11:17:50.199712+0000","last_peered":"2023-11-28T11:17:50.199712+0000","last_clean":"2023-11-28T11:17:50.199712+0000","last_became_active":"2023-11-28T11:17:48.699372+0000","last_became_peered":"2023-11-28T11:17:48.699372+0000","last_unstale":"2023-11-28T11:17:50.199712+0000","last_undegraded":"2023-11-28T11:17:50.199712+0000","last_fullsized":"2023-11-28T11:17:50.199712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:53.370355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200423+0000","last_change":"2023-11-28T11:17:49.212871+0000","last_active":"2023-11-28T11:17:50.200423+0000","last_peered":"2023-11-28T11:17:50.200423+0000","last_clean":"2023-11-28T11:17:50.200423+0000","last_became_active":"2023-11-28T11:17:49.212601+0000","last_became_peered":"2023-11-28T11:17:49.212601+0000","last_unstale":"2023-11-28T11:17:50.200423+0000","last_undegraded":"2023-11-28T11:17:50.200423+0000","last_fullsized":"2023-11-28T11:17:50.200423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199934+0000","last_change":"2023-11-28T11:17:48.697131+0000","last_active":"2023-11-28T11:17:50.199934+0000","last_peered":"2023-11-28T11:17:50.199934+0000","last_clean":"2023-11-28T11:17:50.199934+0000","last_became_active":"2023-11-28T11:17:48.696849+0000","last_became_peered":"2023-11-28T11:17:48.696849+0000","last_unstale":"2023-11-28T11:17:50.199934+0000","last_undegraded":"2023-11-28T11:17:50.199934+0000","last_fullsized":"2023-11-28T11:17:50.199934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:09.707398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200297+0000","last_change":"2023-11-28T11:17:49.216478+0000","last_active":"2023-11-28T11:17:50.200297+0000","last_peered":"2023-11-28T11:17:50.200297+0000","last_clean":"2023-11-28T11:17:50.200297+0000","last_became_active":"2023-11-28T11:17:49.215792+0000","last_became_peered":"2023-11-28T11:17:49.215792+0000","last_unstale":"2023-11-28T11:17:50.200297+0000","last_undegraded":"2023-11-28T11:17:50.200297+0000","last_fullsized":"2023-11-28T11:17:50.200297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970509+0000","last_change":"2023-11-28T11:17:48.692669+0000","last_active":"2023-11-28T11:17:50.970509+0000","last_peered":"2023-11-28T11:17:50.970509+0000","last_clean":"2023-11-28T11:17:50.970509+0000","last_became_active":"2023-11-28T11:17:48.691780+0000","last_became_peered":"2023-11-28T11:17:48.691780+0000","last_unstale":"2023-11-28T11:17:50.970509+0000","last_undegraded":"2023-11-28T11:17:50.970509+0000","last_fullsized":"2023-11-28T11:17:50.970509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970415+0000","last_change":"2023-11-28T11:17:49.220031+0000","last_active":"2023-11-28T11:17:50.970415+0000","last_peered":"2023-11-28T11:17:50.970415+0000","last_clean":"2023-11-28T11:17:50.970415+0000","last_became_active":"2023-11-28T11:17:49.219784+0000","last_became_peered":"2023-11-28T11:17:49.219784+0000","last_unstale":"2023-11-28T11:17:50.970415+0000","last_undegraded":"2023-11-28T11:17:50.970415+0000","last_fullsized":"2023-11-28T11:17:50.970415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:46.899882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.972874+0000","last_change":"2023-11-28T11:17:48.689149+0000","last_active":"2023-11-28T11:17:50.972874+0000","last_peered":"2023-11-28T11:17:50.972874+0000","last_clean":"2023-11-28T11:17:50.972874+0000","last_became_active":"2023-11-28T11:17:48.688826+0000","last_became_peered":"2023-11-28T11:17:48.688826+0000","last_unstale":"2023-11-28T11:17:50.972874+0000","last_undegraded":"2023-11-28T11:17:50.972874+0000","last_fullsized":"2023-11-28T11:17:50.972874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:47.657216+0000","last_clean_scrub_stamp":"2023-11-28T11:17:47.657216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.356420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.199319+0000","last_change":"2023-11-28T11:17:49.208765+0000","last_active":"2023-11-28T11:17:50.199319+0000","last_peered":"2023-11-28T11:17:50.199319+0000","last_clean":"2023-11-28T11:17:50.199319+0000","last_became_active":"2023-11-28T11:17:49.208432+0000","last_became_peered":"2023-11-28T11:17:49.208432+0000","last_unstale":"2023-11-28T11:17:50.199319+0000","last_undegraded":"2023-11-28T11:17:50.199319+0000","last_fullsized":"2023-11-28T11:17:50.199319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.966973+0000","last_change":"2023-11-28T11:17:49.210834+0000","last_active":"2023-11-28T11:17:50.966973+0000","last_peered":"2023-11-28T11:17:50.966973+0000","last_clean":"2023-11-28T11:17:50.966973+0000","last_became_active":"2023-11-28T11:17:49.208613+0000","last_became_peered":"2023-11-28T11:17:49.208613+0000","last_unstale":"2023-11-28T11:17:50.966973+0000","last_undegraded":"2023-11-28T11:17:50.966973+0000","last_fullsized":"2023-11-28T11:17:50.966973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:50.568900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970406+0000","last_change":"2023-11-28T11:17:49.223484+0000","last_active":"2023-11-28T11:17:50.970406+0000","last_peered":"2023-11-28T11:17:50.970406+0000","last_clean":"2023-11-28T11:17:50.970406+0000","last_became_active":"2023-11-28T11:17:49.222011+0000","last_became_peered":"2023-11-28T11:17:49.222011+0000","last_unstale":"2023-11-28T11:17:50.970406+0000","last_undegraded":"2023-11-28T11:17:50.970406+0000","last_fullsized":"2023-11-28T11:17:50.970406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:44.901580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.200166+0000","last_change":"2023-11-28T11:17:49.216669+0000","last_active":"2023-11-28T11:17:50.200166+0000","last_peered":"2023-11-28T11:17:50.200166+0000","last_clean":"2023-11-28T11:17:50.200166+0000","last_became_active":"2023-11-28T11:17:49.216225+0000","last_became_peered":"2023-11-28T11:17:49.216225+0000","last_unstale":"2023-11-28T11:17:50.200166+0000","last_undegraded":"2023-11-28T11:17:50.200166+0000","last_fullsized":"2023-11-28T11:17:50.200166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:22.361832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.970478+0000","last_change":"2023-11-28T11:17:49.219093+0000","last_active":"2023-11-28T11:17:50.970478+0000","last_peered":"2023-11-28T11:17:50.970478+0000","last_clean":"2023-11-28T11:17:50.970478+0000","last_became_active":"2023-11-28T11:17:49.218818+0000","last_became_peered":"2023-11-28T11:17:49.218818+0000","last_unstale":"2023-11-28T11:17:50.970478+0000","last_undegraded":"2023-11-28T11:17:50.970478+0000","last_fullsized":"2023-11-28T11:17:50.970478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:43.609529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:17:50.198448+0000","last_change":"2023-11-28T11:17:49.215688+0000","last_active":"2023-11-28T11:17:50.198448+0000","last_peered":"2023-11-28T11:17:50.198448+0000","last_clean":"2023-11-28T11:17:50.198448+0000","last_became_active":"2023-11-28T11:17:49.215424+0000","last_became_peered":"2023-11-28T11:17:49.215424+0000","last_unstale":"2023-11-28T11:17:50.198448+0000","last_undegraded":"2023-11-28T11:17:50.198448+0000","last_fullsized":"2023-11-28T11:17:50.198448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:48.663432+0000","last_clean_scrub_stamp":"2023-11-28T11:17:48.663432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:38.738948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13188,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358652,"statfs":{"total":96636764160,"available":96623259648,"internally_reserved":0,"allocated":847872,"data_stored":674912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12068,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359772,"statfs":{"total":96636764160,"available":96624406528,"internally_reserved":0,"allocated":225280,"data_stored":82780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":81816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13160,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358680,"statfs":{"total":96636764160,"available":96623288320,"internally_reserved":0,"allocated":819200,"data_stored":672695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12004,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359836,"statfs":{"total":96636764160,"available":96624472064,"internally_reserved":0,"allocated":225280,"data_stored":82780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0510000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12136,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359704,"statfs":{"total":96636764160,"available":96624336896,"internally_reserved":0,"allocated":229376,"data_stored":82870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12152,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359688,"statfs":{"total":96636764160,"available":96624320512,"internally_reserved":0,"allocated":245760,"data_stored":84384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12076,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94359764,"statfs":{"total":96636764160,"available":96624398336,"internally_reserved":0,"allocated":233472,"data_stored":83416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.492}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:18:00.352 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:18:00.352 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:18:00.352 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:18:00.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:18:00.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.586+0000 7fa173dff700 1 Processor -- start 2023-11-28T11:18:00.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.586+0000 7fa173dff700 1 -- start start 2023-11-28T11:18:00.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.587+0000 7fa173dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa16c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.587+0000 7fa173dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa16c0b20b0 0x7fa16c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.587+0000 7fa173dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c0b0230 con 0x7fa16c0d01b0 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.587+0000 7fa173dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c15cec0 con 0x7fa16c0b20b0 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.587+0000 7fa173dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b4570 con 0x7fa16c0d1330 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa171b9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa16c0d01b0 0x7fa16c0d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46984/0 (socket says 172.21.15.53:46984) 2023-11-28T11:18:00.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa171b9b700 1 -- 172.21.15.53:0/604520495 learned_addr learned my addr 172.21.15.53:0/604520495 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa17139a700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa16c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa17239c700 1 --2- 172.21.15.53:0/604520495 >> v2:172.21.15.157:3300/0 conn(0x7fa16c0b20b0 0x7fa16c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266188502 0 0) 0x7fa16c0b0230 con 0x7fa16c0d01b0 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1580035b0 con 0x7fa16c0d01b0 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300521245 0 0) 0x7fa1580035b0 con 0x7fa16c0d01b0 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 msgr2=0x7fa16c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa16c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 >> v2:172.21.15.157:3300/0 conn(0x7fa16c0b20b0 msgr2=0x7fa16c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 --2- 172.21.15.53:0/604520495 >> v2:172.21.15.157:3300/0 conn(0x7fa16c0b20b0 0x7fa16c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.588+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0b4850 con 0x7fa16c0d01b0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa168001670 con 0x7fa16c0d01b0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 >> v1:172.21.15.53:6789/0 conn(0x7fa16c0d01b0 legacy=0x7fa16c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa158003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa158003fd0 0x7fa158004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa158004d90 0x7fa158005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c15cec0 con 0x7fa158003fd0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b4570 con 0x7fa158004d90 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800ba10 con 0x7fa16c0d1330 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa158003fd0 0x7fa158004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa171b9b700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa158003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17239c700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa158004d90 0x7fa158005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 -- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 msgr2=0x7fa158003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa16c0d1330 0x7fa158003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 -- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa158004d90 msgr2=0x7fa158005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa158004d90 0x7fa158005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17139a700 1 -- 172.21.15.53:0/604520495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580035b0 con 0x7fa158003fd0 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.589+0000 7fa17239c700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa158004d90 0x7fa158005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa17139a700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa158003fd0 0x7fa158004850 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009b30 tx=0x7fa158005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c1791236e2d35c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e6c0 con 0x7fa158003fd0 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00e820 con 0x7fa158003fd0 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa170b99700 1 -- 172.21.15.53:0/604520495 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00eb90 con 0x7fa158003fd0 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa173dff700 1 -- 172.21.15.53:0/604520495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa158003fd0 msgr2=0x7fa158004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa173dff700 1 --2- 172.21.15.53:0/604520495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa158003fd0 0x7fa158004850 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009b30 tx=0x7fa158005880 comp rx=0 tx=0).stop 2023-11-28T11:18:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa17239c700 1 -- 172.21.15.53:0/604520495 reap_dead start 2023-11-28T11:18:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa173dff700 1 -- 172.21.15.53:0/604520495 shutdown_connections 2023-11-28T11:18:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa173dff700 1 -- 172.21.15.53:0/604520495 >> 172.21.15.53:0/604520495 conn(0x7fa16c09c3f0 msgr2=0x7fa16c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.590+0000 7fa173dff700 1 -- 172.21.15.53:0/604520495 shutdown_connections 2023-11-28T11:18:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.591+0000 7fa173dff700 1 -- 172.21.15.53:0/604520495 wait complete. 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.592+0000 7fa173dff700 1 Processor -- start 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.592+0000 7fa173dff700 1 -- start start 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 0x7fa16c06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 0x7fa16c0a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cbfa0 con 0x7fa16c0cae90 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cb320 con 0x7fa16c0cf550 2023-11-28T11:18:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.593+0000 7fa173dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cb600 con 0x7fa158003fd0 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40292/0 (socket says 172.21.15.53:40292) 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 -- 172.21.15.53:0/1283181776 learned_addr learned my addr 172.21.15.53:0/1283181776 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa171b9b700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 0x7fa16c06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 -- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 msgr2=0x7fa16c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17239c700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 0x7fa16c0a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 0x7fa16c06b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 -- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 msgr2=0x7fa16c0a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 0x7fa16c0a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 -- 172.21.15.53:0/1283181776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0afec0 con 0x7fa16c0cae90 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa171b9b700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 0x7fa16c06b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17239c700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 0x7fa16c0a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.594+0000 7fa17139a700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0090b0 tx=0x7fa15c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.595+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e6c0 con 0x7fa16c0cae90 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.595+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c003ad0 con 0x7fa16c0cae90 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.595+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c0afbe0 con 0x7fa16c0cae90 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.595+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0045d0 con 0x7fa16c0cae90 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.595+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c0b2600 con 0x7fa16c0cae90 2023-11-28T11:18:00.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.596+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c068690 con 0x7fa16c0cae90 2023-11-28T11:18:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.597+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa15c025070 con 0x7fa16c0cae90 2023-11-28T11:18:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.597+0000 7fa1627fc700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 0x7fa15006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.598+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa15c0875d0 con 0x7fa16c0cae90 2023-11-28T11:18:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.598+0000 7fa171b9b700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 0x7fa15006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.599+0000 7fa171b9b700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 0x7fa15006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa168000fb0 tx=0x7fa168009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:00.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.602+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa15c053d40 con 0x7fa16c0cae90 2023-11-28T11:18:00.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.851+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa16c0b1270 con 0x7fa16c0cae90 2023-11-28T11:18:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.851+0000 7fa1627fc700 1 -- 172.21.15.53:0/1283181776 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa15c057360 con 0x7fa16c0cae90 2023-11-28T11:18:00.897 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:00.897 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 msgr2=0x7fa15006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 0x7fa15006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa168000fb0 tx=0x7fa168009040 comp rx=0 tx=0).stop 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 msgr2=0x7fa16c09ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0090b0 tx=0x7fa15c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 shutdown_connections 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa1500682f0 0x7fa15006a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa158003fd0 0x7fa16c06b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16c0cf550 0x7fa16c0a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 --2- 172.21.15.53:0/1283181776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa16c0cae90 0x7fa16c09ff40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:00.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 >> 172.21.15.53:0/1283181776 conn(0x7fa16c09c3f0 msgr2=0x7fa16c0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:00.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 shutdown_connections 2023-11-28T11:18:00.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:00.854+0000 7fa173dff700 1 -- 172.21.15.53:0/1283181776 wait complete. 2023-11-28T11:18:00.900 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:18:00.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.100+0000 7efc34bff700 1 Processor -- start 2023-11-28T11:18:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.101+0000 7efc34bff700 1 -- start start 2023-11-28T11:18:01.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc34bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 0x7efc300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc34bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc30161340 0x7efc300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc34bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300d1950 con 0x7efc300d2320 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc34bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300b5800 con 0x7efc30161340 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc34bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300b5520 con 0x7efc3015c9b0 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc30161340 0x7efc300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc30161340 0x7efc300d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45610/0 (socket says 172.21.15.53:45610) 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 -- 172.21.15.53:0/12178022 learned_addr learned my addr 172.21.15.53:0/12178022 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc2e59c700 1 --2- 172.21.15.53:0/12178022 >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 0x7efc300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 -- 172.21.15.53:0/12178022 >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 msgr2=0x7efc300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 --2- 172.21.15.53:0/12178022 >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 0x7efc300b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 -- 172.21.15.53:0/12178022 >> v1:172.21.15.157:6789/0 conn(0x7efc300d2320 legacy=0x7efc300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc27fff700 1 -- 172.21.15.53:0/12178022 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300d1300 con 0x7efc30161340 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.102+0000 7efc2e59c700 1 --2- 172.21.15.53:0/12178022 >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 0x7efc300b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc27fff700 1 --2- 172.21.15.53:0/12178022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc30161340 0x7efc300d1de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc30067460 tx=0x7efc18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fe03af5952f616 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc2dd9b700 1 -- 172.21.15.53:0/12178022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18004230 con 0x7efc30161340 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc2dd9b700 1 -- 172.21.15.53:0/12178022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18004390 con 0x7efc30161340 2023-11-28T11:18:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc2dd9b700 1 -- 172.21.15.53:0/12178022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180036a0 con 0x7efc30161340 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc34bff700 1 -- 172.21.15.53:0/12178022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc30161340 msgr2=0x7efc300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.103+0000 7efc34bff700 1 --2- 172.21.15.53:0/12178022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc30161340 0x7efc300d1de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc30067460 tx=0x7efc18009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 -- 172.21.15.53:0/12178022 shutdown_connections 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 --2- 172.21.15.53:0/12178022 >> v2:172.21.15.157:3300/0 conn(0x7efc3015c9b0 0x7efc300b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 --2- 172.21.15.53:0/12178022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc30161340 0x7efc300d1de0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 -- 172.21.15.53:0/12178022 >> 172.21.15.53:0/12178022 conn(0x7efc30072600 msgr2=0x7efc300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 -- 172.21.15.53:0/12178022 shutdown_connections 2023-11-28T11:18:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.104+0000 7efc34bff700 1 -- 172.21.15.53:0/12178022 wait complete. 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.106+0000 7efc34bff700 1 Processor -- start 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.106+0000 7efc34bff700 1 -- start start 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 0x7efc300c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc30161340 0x7efc30146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c8840 con 0x7efc3015c9b0 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c8b20 con 0x7efc300d2320 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc34bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c8e00 con 0x7efc30161340 2023-11-28T11:18:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45620/0 (socket says 172.21.15.53:45620) 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 -- 172.21.15.53:0/1510489047 learned_addr learned my addr 172.21.15.53:0/1510489047 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc2e59c700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 0x7efc300c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc2ed9d700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc30161340 0x7efc30146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 -- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc30161340 msgr2=0x7efc30146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc30161340 0x7efc30146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 -- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 msgr2=0x7efc300c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 0x7efc300c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc27fff700 1 -- 172.21.15.53:0/1510489047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009550 con 0x7efc3015c9b0 2023-11-28T11:18:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.107+0000 7efc2e59c700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 0x7efc300c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc27fff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc18009130 tx=0x7efc18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18003dc0 con 0x7efc3015c9b0 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1801f070 con 0x7efc3015c9b0 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011870 con 0x7efc3015c9b0 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300b5130 con 0x7efc3015c9b0 2023-11-28T11:18:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.108+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc300d15e0 con 0x7efc3015c9b0 2023-11-28T11:18:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.109+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300b52c0 con 0x7efc3015c9b0 2023-11-28T11:18:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.110+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc18017020 con 0x7efc3015c9b0 2023-11-28T11:18:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.111+0000 7efc26ffd700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 0x7efc1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.111+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc18087620 con 0x7efc3015c9b0 2023-11-28T11:18:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.111+0000 7efc2e59c700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 0x7efc1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.112+0000 7efc2e59c700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 0x7efc1006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc20006fd0 tx=0x7efc20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.115+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc18053d90 con 0x7efc3015c9b0 2023-11-28T11:18:01.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.320+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc300745c0 con 0x7efc3015c9b0 2023-11-28T11:18:01.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.322+0000 7efc26ffd700 1 -- 172.21.15.53:0/1510489047 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7efc180573b0 con 0x7efc3015c9b0 2023-11-28T11:18:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:01.323 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:01.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 msgr2=0x7efc1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 0x7efc1006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc20006fd0 tx=0x7efc20008040 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 msgr2=0x7efc30150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc18009130 tx=0x7efc18004750 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 shutdown_connections 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc100682f0 0x7efc1006a7a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc30161340 0x7efc30146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc300d2320 0x7efc300c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 --2- 172.21.15.53:0/1510489047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3015c9b0 0x7efc30150830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 >> 172.21.15.53:0/1510489047 conn(0x7efc30072600 msgr2=0x7efc3015cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 shutdown_connections 2023-11-28T11:18:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.324+0000 7efc34bff700 1 -- 172.21.15.53:0/1510489047 wait complete. 2023-11-28T11:18:01.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:01.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:01.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.546+0000 7ff968fff700 1 Processor -- start 2023-11-28T11:18:01.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.546+0000 7ff968fff700 1 -- start start 2023-11-28T11:18:01.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.547+0000 7ff968fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.547+0000 7ff968fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff9640d2320 0x7ff9640cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.547+0000 7ff968fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9640d1950 con 0x7ff96415c9b0 2023-11-28T11:18:01.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.547+0000 7ff968fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9640b5800 con 0x7ff9640d2320 2023-11-28T11:18:01.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.547+0000 7ff968fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640b5300 con 0x7ff964161340 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff9640d2320 0x7ff9640cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff9640d2320 0x7ff9640cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45632/0 (socket says 172.21.15.53:45632) 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 -- 172.21.15.53:0/3106745776 learned_addr learned my addr 172.21.15.53:0/3106745776 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff961d9b700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 -- 172.21.15.53:0/3106745776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 msgr2=0x7ff9640d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9640d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 -- 172.21.15.53:0/3106745776 >> v1:172.21.15.157:6789/0 conn(0x7ff96415c9b0 legacy=0x7ff9640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 -- 172.21.15.53:0/3106745776 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640b5520 con 0x7ff9640d2320 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff962d9d700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640cd7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009e80 tx=0x7ff95c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa3e2f5a2870369 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff96159a700 1 -- 172.21.15.53:0/3106745776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00e070 con 0x7ff9640d2320 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff96159a700 1 -- 172.21.15.53:0/3106745776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00b980 con 0x7ff9640d2320 2023-11-28T11:18:01.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.548+0000 7ff96159a700 1 -- 172.21.15.53:0/3106745776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00bd30 con 0x7ff9640d2320 2023-11-28T11:18:01.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 -- 172.21.15.53:0/3106745776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 msgr2=0x7ff9640cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640cd7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009e80 tx=0x7ff95c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 -- 172.21.15.53:0/3106745776 shutdown_connections 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640cd7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 --2- 172.21.15.53:0/3106745776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9640d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 -- 172.21.15.53:0/3106745776 >> 172.21.15.53:0/3106745776 conn(0x7ff964072600 msgr2=0x7ff9640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 -- 172.21.15.53:0/3106745776 shutdown_connections 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.549+0000 7ff968fff700 1 -- 172.21.15.53:0/3106745776 wait complete. 2023-11-28T11:18:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.551+0000 7ff968fff700 1 Processor -- start 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.551+0000 7ff968fff700 1 -- start start 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 0x7ff9640c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9641f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640c6ca0 con 0x7ff9640d2320 2023-11-28T11:18:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641f3110 con 0x7ff96415c9b0 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff968fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641f33f0 con 0x7ff964161340 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff96259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff96259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45636/0 (socket says 172.21.15.53:45636) 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff96259c700 1 -- 172.21.15.53:0/1066194075 learned_addr learned my addr 172.21.15.53:0/1066194075 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff962d9d700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9641f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.552+0000 7ff961d9b700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 0x7ff9640c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 -- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 msgr2=0x7ff9641f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9641f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 -- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 msgr2=0x7ff9640c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 0x7ff9640c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 -- 172.21.15.53:0/1066194075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009b30 con 0x7ff9640d2320 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff961d9b700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 0x7ff9640c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:01.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff96259c700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c450 tx=0x7ff95400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540052d0 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff954005430 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540038c0 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9640d15e0 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.553+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9640d1300 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.554+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96415cd80 con 0x7ff9640d2320 2023-11-28T11:18:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.555+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff954019030 con 0x7ff9640d2320 2023-11-28T11:18:01.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.556+0000 7ff952ffd700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.556+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff954084f10 con 0x7ff9640d2320 2023-11-28T11:18:01.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.557+0000 7ff961d9b700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.558+0000 7ff961d9b700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 0x7ff94406a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9640b7990 tx=0x7ff958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.561+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff954051770 con 0x7ff9640d2320 2023-11-28T11:18:01.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.765+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9640745c0 con 0x7ff9640d2320 2023-11-28T11:18:01.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.765+0000 7ff952ffd700 1 -- 172.21.15.53:0/1066194075 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9540170a0 con 0x7ff9640d2320 2023-11-28T11:18:01.766 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 msgr2=0x7ff94406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 0x7ff94406a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9640b7990 tx=0x7ff958008040 comp rx=0 tx=0).stop 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 msgr2=0x7ff9640c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c450 tx=0x7ff95400c760 comp rx=0 tx=0).stop 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 shutdown_connections 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff964161340 0x7ff9641f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9640d2320 0x7ff9640c94d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 --2- 172.21.15.53:0/1066194075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96415c9b0 0x7ff9640c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 >> 172.21.15.53:0/1066194075 conn(0x7ff964072600 msgr2=0x7ff9640b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 shutdown_connections 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.768+0000 7ff968fff700 1 -- 172.21.15.53:0/1066194075 wait complete. 2023-11-28T11:18:01.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:01.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.991+0000 7f097e7ff700 1 Processor -- start 2023-11-28T11:18:01.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.991+0000 7f097e7ff700 1 -- start start 2023-11-28T11:18:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f097815c9b0 0x7f09780b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0978161340 0x7f09780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09780d2320 0x7f09780cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09780b5800 con 0x7f097815c9b0 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09780d1950 con 0x7f09780d2320 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f097e7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09780b5520 con 0x7f0978161340 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f0977fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f097815c9b0 0x7f09780b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.992+0000 7f0977fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f097815c9b0 0x7f09780b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45648/0 (socket says 172.21.15.53:45648) 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 -- 172.21.15.53:0/4184518976 learned_addr learned my addr 172.21.15.53:0/4184518976 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f09777fe700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0978161340 0x7f09780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f097cd9c700 1 --2- 172.21.15.53:0/4184518976 >> v2:172.21.15.157:3300/0 conn(0x7f09780d2320 0x7f09780cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 -- 172.21.15.53:0/4184518976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0978161340 msgr2=0x7f09780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0978161340 0x7f09780d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 -- 172.21.15.53:0/4184518976 >> v2:172.21.15.157:3300/0 conn(0x7f09780d2320 msgr2=0x7f09780cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 --2- 172.21.15.53:0/4184518976 >> v2:172.21.15.157:3300/0 conn(0x7f09780d2320 0x7f09780cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 -- 172.21.15.53:0/4184518976 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09780d1300 con 0x7f097815c9b0 2023-11-28T11:18:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0977fff700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f097815c9b0 0x7f09780b4bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f097808c4d0 tx=0x7f0968009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d904ea643f80abc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0976ffd700 1 -- 172.21.15.53:0/4184518976 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968011830 con 0x7f097815c9b0 2023-11-28T11:18:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.993+0000 7f0976ffd700 1 -- 172.21.15.53:0/4184518976 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968011990 con 0x7f097815c9b0 2023-11-28T11:18:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f0976ffd700 1 -- 172.21.15.53:0/4184518976 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968011d20 con 0x7f097815c9b0 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 -- 172.21.15.53:0/4184518976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f097815c9b0 msgr2=0x7f09780b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f097815c9b0 0x7f09780b4bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f097808c4d0 tx=0x7f0968009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 -- 172.21.15.53:0/4184518976 shutdown_connections 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 --2- 172.21.15.53:0/4184518976 >> v2:172.21.15.157:3300/0 conn(0x7f09780d2320 0x7f09780cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f097815c9b0 0x7f09780b4bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 --2- 172.21.15.53:0/4184518976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0978161340 0x7f09780d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 -- 172.21.15.53:0/4184518976 >> 172.21.15.53:0/4184518976 conn(0x7f0978072600 msgr2=0x7f09780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 -- 172.21.15.53:0/4184518976 shutdown_connections 2023-11-28T11:18:01.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.994+0000 7f097e7ff700 1 -- 172.21.15.53:0/4184518976 wait complete. 2023-11-28T11:18:01.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.996+0000 7f097e7ff700 1 Processor -- start 2023-11-28T11:18:01.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.996+0000 7f097e7ff700 1 -- start start 2023-11-28T11:18:01.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:01.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f097815c9b0 0x7f09781f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 0x7f09781f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097814ee70 con 0x7f0978161340 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09781e7ba0 con 0x7f09780d2320 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097e7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09781f31a0 con 0x7f097815c9b0 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f0977fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f097cd9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 0x7f09781f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f0977fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:33260/0 (socket says 172.21.15.53:33260) 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f0977fff700 1 -- 172.21.15.53:0/1626999500 learned_addr learned my addr 172.21.15.53:0/1626999500 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.997+0000 7f09777fe700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f097815c9b0 0x7f09781f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 -- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f097815c9b0 msgr2=0x7f09781f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f097815c9b0 0x7f09781f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 -- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 msgr2=0x7f09781f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 -- 172.21.15.53:0/1626999500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968009550 con 0x7f0978161340 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f0977fff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097cd9c700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 0x7f09781f39f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f097000eba0 tx=0x7f097000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970009a80 con 0x7f0978161340 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097000f040 con 0x7f0978161340 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000cc40 con 0x7f0978161340 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09780b5130 con 0x7f0978161340 2023-11-28T11:18:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.998+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09780d15e0 con 0x7f0978161340 2023-11-28T11:18:02.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:01.999+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09780b52c0 con 0x7f0978161340 2023-11-28T11:18:02.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.000+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0970014480 con 0x7f0978161340 2023-11-28T11:18:02.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.001+0000 7f0974ff9700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 0x7f096406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.001+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09700867e0 con 0x7f0978161340 2023-11-28T11:18:02.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.001+0000 7f0977fff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 0x7f096406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.002+0000 7f0977fff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 0x7f096406a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0968009130 tx=0x7f0968008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.005+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0970052ef0 con 0x7f0978161340 2023-11-28T11:18:02.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.206+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09780745c0 con 0x7f0978161340 2023-11-28T11:18:02.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.206+0000 7f0974ff9700 1 -- 172.21.15.53:0/1626999500 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0970018020 con 0x7f0978161340 2023-11-28T11:18:02.207 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.208+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 msgr2=0x7f096406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.208+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 0x7f096406a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0968009130 tx=0x7f0968008030 comp rx=0 tx=0).stop 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.208+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 msgr2=0x7f09781f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.208+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 0x7f09781f39f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f097000eba0 tx=0x7f097000eeb0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 shutdown_connections 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f09640682f0 0x7f096406a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0978161340 0x7f09781f39f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09780d2320 0x7f09781f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 --2- 172.21.15.53:0/1626999500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f097815c9b0 0x7f09781f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 >> 172.21.15.53:0/1626999500 conn(0x7f0978072600 msgr2=0x7f09780b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:02.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 shutdown_connections 2023-11-28T11:18:02.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.209+0000 7f097e7ff700 1 -- 172.21.15.53:0/1626999500 wait complete. 2023-11-28T11:18:02.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.431+0000 7fd0211ff700 1 Processor -- start 2023-11-28T11:18:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.431+0000 7fd0211ff700 1 -- start start 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.432+0000 7fd0211ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd01c15c9b0 0x7fd01c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.432+0000 7fd0211ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c161340 0x7fd01c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.432+0000 7fd0211ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c0d1950 con 0x7fd01c0d2320 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.432+0000 7fd0211ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c0b5800 con 0x7fd01c15c9b0 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.432+0000 7fd0211ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0b5520 con 0x7fd01c161340 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01b59e700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd01c0d2320 0x7fd01c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55818/0 (socket says 172.21.15.53:55818) 2023-11-28T11:18:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01b59e700 1 -- 172.21.15.53:0/2410448058 learned_addr learned my addr 172.21.15.53:0/2410448058 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/2410448058 >> v2:172.21.15.157:3300/0 conn(0x7fd01c15c9b0 0x7fd01c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01a59c700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c161340 0x7fd01c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd019d9b700 1 -- 172.21.15.53:0/2410448058 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301597791 0 0) 0x7fd01c0d1950 con 0x7fd01c0d2320 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd019d9b700 1 -- 172.21.15.53:0/2410448058 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0040035b0 con 0x7fd01c0d2320 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 -- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c161340 msgr2=0x7fd01c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c161340 0x7fd01c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 -- 172.21.15.53:0/2410448058 >> v1:172.21.15.53:6789/0 conn(0x7fd01c0d2320 legacy=0x7fd01c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 -- 172.21.15.53:0/2410448058 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0d1300 con 0x7fd01c15c9b0 2023-11-28T11:18:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.433+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c15c9b0 0x7fd01c0b4bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009840 tx=0x7fd00c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4910f38ecc34b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd019d9b700 1 -- 172.21.15.53:0/2410448058 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c011830 con 0x7fd01c15c9b0 2023-11-28T11:18:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd019d9b700 1 -- 172.21.15.53:0/2410448058 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c011990 con 0x7fd01c15c9b0 2023-11-28T11:18:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd019d9b700 1 -- 172.21.15.53:0/2410448058 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c011d00 con 0x7fd01c15c9b0 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 -- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c15c9b0 msgr2=0x7fd01c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c15c9b0 0x7fd01c0b4bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009840 tx=0x7fd00c009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 -- 172.21.15.53:0/2410448058 shutdown_connections 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c15c9b0 0x7fd01c0b4bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 --2- 172.21.15.53:0/2410448058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c161340 0x7fd01c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.434+0000 7fd0211ff700 1 -- 172.21.15.53:0/2410448058 >> 172.21.15.53:0/2410448058 conn(0x7fd01c072600 msgr2=0x7fd01c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.435+0000 7fd0211ff700 1 -- 172.21.15.53:0/2410448058 shutdown_connections 2023-11-28T11:18:02.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.435+0000 7fd0211ff700 1 -- 172.21.15.53:0/2410448058 wait complete. 2023-11-28T11:18:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.436+0000 7fd0211ff700 1 Processor -- start 2023-11-28T11:18:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.436+0000 7fd0211ff700 1 -- start start 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c0d2320 0x7fd01c1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c15c9b0 0x7fd01c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c151570 con 0x7fd01c161340 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c146e00 con 0x7fd01c0d2320 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.437+0000 7fd0211ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c157670 con 0x7fd01c15c9b0 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45668/0 (socket says 172.21.15.53:45668) 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 -- 172.21.15.53:0/3695075837 learned_addr learned my addr 172.21.15.53:0/3695075837 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c0d2320 0x7fd01c1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01a59c700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c15c9b0 0x7fd01c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 -- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c15c9b0 msgr2=0x7fd01c0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c15c9b0 0x7fd01c0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 -- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c0d2320 msgr2=0x7fd01c1468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c0d2320 0x7fd01c1468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 -- 172.21.15.53:0/3695075837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0094a0 con 0x7fd01c161340 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.438+0000 7fd01b59e700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd01400cad0 tx=0x7fd01400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.439+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400a9a0 con 0x7fd01c161340 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.439+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01400ab00 con 0x7fd01c161340 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.439+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400ae90 con 0x7fd01c161340 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.439+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c0b5130 con 0x7fd01c161340 2023-11-28T11:18:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.439+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c0d15e0 con 0x7fd01c161340 2023-11-28T11:18:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.440+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c0b52c0 con 0x7fd01c161340 2023-11-28T11:18:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.441+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd014019030 con 0x7fd01c161340 2023-11-28T11:18:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.441+0000 7fd0037fe700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 0x7fd00806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.442+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd014085820 con 0x7fd01c161340 2023-11-28T11:18:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.442+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 0x7fd00806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.442+0000 7fd01ad9d700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 0x7fd00806a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009fd0 tx=0x7fd00c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.445+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd014051f90 con 0x7fd01c161340 2023-11-28T11:18:02.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.645+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd01c0745c0 con 0x7fd01c161340 2023-11-28T11:18:02.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.646+0000 7fd0037fe700 1 -- 172.21.15.53:0/3695075837 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0140555b0 con 0x7fd01c161340 2023-11-28T11:18:02.647 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:02.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 msgr2=0x7fd00806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 0x7fd00806a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009fd0 tx=0x7fd00c01c040 comp rx=0 tx=0).stop 2023-11-28T11:18:02.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 msgr2=0x7fd01c0b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd01400cad0 tx=0x7fd01400cde0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 shutdown_connections 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd0080682f0 0x7fd00806a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd01c161340 0x7fd01c0b7ae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd01c0d2320 0x7fd01c1468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 --2- 172.21.15.53:0/3695075837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd01c15c9b0 0x7fd01c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 >> 172.21.15.53:0/3695075837 conn(0x7fd01c072600 msgr2=0x7fd01c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 shutdown_connections 2023-11-28T11:18:02.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.648+0000 7fd0211ff700 1 -- 172.21.15.53:0/3695075837 wait complete. 2023-11-28T11:18:02.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:02.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.871+0000 7fe4e77ff700 1 Processor -- start 2023-11-28T11:18:02.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.871+0000 7fe4e77ff700 1 -- start start 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 0x7fe4e00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e0161340 0x7fe4e00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00b5800 con 0x7fe4e0161340 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00d1950 con 0x7fe4e015c9b0 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00b5520 con 0x7fe4e00d2320 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e0161340 0x7fe4e00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e0161340 0x7fe4e00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45684/0 (socket says 172.21.15.53:45684) 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/2921065 learned_addr learned my addr 172.21.15.53:0/2921065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:02.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/2921065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 msgr2=0x7fe4e00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e5d9c700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e559b700 1 --2- 172.21.15.53:0/2921065 >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 0x7fe4e00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/2921065 >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 msgr2=0x7fe4e00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/2921065 >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 0x7fe4e00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/2921065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00d1300 con 0x7fe4e0161340 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e559b700 1 --2- 172.21.15.53:0/2921065 >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 0x7fe4e00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.873+0000 7fe4e5d9c700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e0161340 0x7fe4e00d1de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0067460 tx=0x7fe4d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3b92c2d95578a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4d7fff700 1 -- 172.21.15.53:0/2921065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000b870 con 0x7fe4e0161340 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4d7fff700 1 -- 172.21.15.53:0/2921065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000b9d0 con 0x7fe4e0161340 2023-11-28T11:18:02.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4d7fff700 1 -- 172.21.15.53:0/2921065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000bd60 con 0x7fe4e0161340 2023-11-28T11:18:02.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4e77ff700 1 -- 172.21.15.53:0/2921065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e0161340 msgr2=0x7fe4e00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e0161340 0x7fe4e00d1de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0067460 tx=0x7fe4d0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.874+0000 7fe4e77ff700 1 -- 172.21.15.53:0/2921065 shutdown_connections 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/2921065 >> v2:172.21.15.157:3300/0 conn(0x7fe4e015c9b0 0x7fe4e00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/2921065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e0161340 0x7fe4e00d1de0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 -- 172.21.15.53:0/2921065 >> 172.21.15.53:0/2921065 conn(0x7fe4e0072600 msgr2=0x7fe4e00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 -- 172.21.15.53:0/2921065 shutdown_connections 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.875+0000 7fe4e77ff700 1 -- 172.21.15.53:0/2921065 wait complete. 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.876+0000 7fe4e77ff700 1 Processor -- start 2023-11-28T11:18:02.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.876+0000 7fe4e77ff700 1 -- start start 2023-11-28T11:18:02.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 0x7fe4e01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01f5490 con 0x7fe4e015c9b0 2023-11-28T11:18:02.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01f4880 con 0x7fe4e0161340 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.877+0000 7fe4e77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01f4b60 con 0x7fe4e00d2320 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45692/0 (socket says 172.21.15.53:45692) 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/142840294 learned_addr learned my addr 172.21.15.53:0/142840294 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e559b700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e5d9c700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 0x7fe4e01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 msgr2=0x7fe4e00c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 msgr2=0x7fe4e01594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 0x7fe4e01594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 -- 172.21.15.53:0/142840294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009550 con 0x7fe4e015c9b0 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e559b700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:02.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e5d9c700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 0x7fe4e01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4e4d9a700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00675d0 tx=0x7fe4d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.878+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004310 con 0x7fe4e015c9b0 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.879+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d001f070 con 0x7fe4e015c9b0 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.879+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00b5130 con 0x7fe4e015c9b0 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.879+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00036a0 con 0x7fe4e015c9b0 2023-11-28T11:18:02.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.879+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e00d15e0 con 0x7fe4e015c9b0 2023-11-28T11:18:02.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.880+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e00b52c0 con 0x7fe4e015c9b0 2023-11-28T11:18:02.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.881+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4d002a020 con 0x7fe4e015c9b0 2023-11-28T11:18:02.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.882+0000 7fe4d5ffb700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 0x7fe4c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:02.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.882+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4d00873c0 con 0x7fe4e015c9b0 2023-11-28T11:18:02.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.882+0000 7fe4e559b700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 0x7fe4c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:02.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.883+0000 7fe4e559b700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 0x7fe4c406a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006fd0 tx=0x7fe4dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:02.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:02.886+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4d0053b30 con 0x7fe4e015c9b0 2023-11-28T11:18:03.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.089+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4e00745c0 con 0x7fe4e015c9b0 2023-11-28T11:18:03.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.089+0000 7fe4d5ffb700 1 -- 172.21.15.53:0/142840294 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4d0057150 con 0x7fe4e015c9b0 2023-11-28T11:18:03.090 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 msgr2=0x7fe4c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 0x7fe4c406a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006fd0 tx=0x7fe4dc008040 comp rx=0 tx=0).stop 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 msgr2=0x7fe4e0158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00675d0 tx=0x7fe4d0004750 comp rx=0 tx=0).stop 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 shutdown_connections 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe4c40682f0 0x7fe4c406a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4e00d2320 0x7fe4e00c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4e0161340 0x7fe4e01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 --2- 172.21.15.53:0/142840294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e015c9b0 0x7fe4e0158f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.091+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 >> 172.21.15.53:0/142840294 conn(0x7fe4e0072600 msgr2=0x7fe4e00b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:03.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.092+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 shutdown_connections 2023-11-28T11:18:03.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.092+0000 7fe4e77ff700 1 -- 172.21.15.53:0/142840294 wait complete. 2023-11-28T11:18:03.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:18:03.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.311+0000 7fdba21ff700 1 Processor -- start 2023-11-28T11:18:03.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.311+0000 7fdba21ff700 1 -- start start 2023-11-28T11:18:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdba21ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb9c15c9b0 0x7fdb9c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdba21ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb9c161340 0x7fdb9c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdba21ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0d1950 con 0x7fdb9c0d2320 2023-11-28T11:18:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdba21ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b5800 con 0x7fdb9c161340 2023-11-28T11:18:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdba21ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b5520 con 0x7fdb9c15c9b0 2023-11-28T11:18:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdb9bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdb9c0d2320 0x7fdb9c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55828/0 (socket says 172.21.15.53:55828) 2023-11-28T11:18:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdb9bfff700 1 -- 172.21.15.53:0/3571340887 learned_addr learned my addr 172.21.15.53:0/3571340887 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdb9affd700 1 --2- 172.21.15.53:0/3571340887 >> v2:172.21.15.53:3300/0 conn(0x7fdb9c161340 0x7fdb9c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.312+0000 7fdb9b7fe700 1 --2- 172.21.15.53:0/3571340887 >> v2:172.21.15.157:3300/0 conn(0x7fdb9c15c9b0 0x7fdb9c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9a7fc700 1 -- 172.21.15.53:0/3571340887 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758866053 0 0) 0x7fdb9c0d1950 con 0x7fdb9c0d2320 2023-11-28T11:18:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9a7fc700 1 -- 172.21.15.53:0/3571340887 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb840035b0 con 0x7fdb9c0d2320 2023-11-28T11:18:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9affd700 1 -- 172.21.15.53:0/3571340887 >> v2:172.21.15.157:3300/0 conn(0x7fdb9c15c9b0 msgr2=0x7fdb9c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9affd700 1 --2- 172.21.15.53:0/3571340887 >> v2:172.21.15.157:3300/0 conn(0x7fdb9c15c9b0 0x7fdb9c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9affd700 1 -- 172.21.15.53:0/3571340887 >> v1:172.21.15.53:6789/0 conn(0x7fdb9c0d2320 legacy=0x7fdb9c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9affd700 1 -- 172.21.15.53:0/3571340887 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0d1300 con 0x7fdb9c161340 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9affd700 1 --2- 172.21.15.53:0/3571340887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c0d1de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c067460 tx=0x7fdb90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c93f57bd965c596 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9a7fc700 1 -- 172.21.15.53:0/3571340887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90011830 con 0x7fdb9c161340 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9a7fc700 1 -- 172.21.15.53:0/3571340887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb90011990 con 0x7fdb9c161340 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.313+0000 7fdb9a7fc700 1 -- 172.21.15.53:0/3571340887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90011d20 con 0x7fdb9c161340 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 -- 172.21.15.53:0/3571340887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 msgr2=0x7fdb9c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 --2- 172.21.15.53:0/3571340887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c0d1de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c067460 tx=0x7fdb90009b20 comp rx=0 tx=0).stop 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 -- 172.21.15.53:0/3571340887 shutdown_connections 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 --2- 172.21.15.53:0/3571340887 >> v2:172.21.15.157:3300/0 conn(0x7fdb9c15c9b0 0x7fdb9c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 --2- 172.21.15.53:0/3571340887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c0d1de0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 -- 172.21.15.53:0/3571340887 >> 172.21.15.53:0/3571340887 conn(0x7fdb9c072600 msgr2=0x7fdb9c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:03.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 -- 172.21.15.53:0/3571340887 shutdown_connections 2023-11-28T11:18:03.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.314+0000 7fdba21ff700 1 -- 172.21.15.53:0/3571340887 wait complete. 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.316+0000 7fdba21ff700 1 Processor -- start 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.316+0000 7fdba21ff700 1 -- start start 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9c0d2320 0x7fdb9c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9c15c9b0 0x7fdb9c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c155b30 con 0x7fdb9c161340 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c154f20 con 0x7fdb9c15c9b0 2023-11-28T11:18:03.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdba21ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c155200 con 0x7fdb9c0d2320 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45704/0 (socket says 172.21.15.53:45704) 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 -- 172.21.15.53:0/2841367377 learned_addr learned my addr 172.21.15.53:0/2841367377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9b7fe700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9c0d2320 0x7fdb9c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9affd700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9c15c9b0 0x7fdb9c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 -- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9c0d2320 msgr2=0x7fdb9c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9c0d2320 0x7fdb9c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 -- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9c15c9b0 msgr2=0x7fdb9c14d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9c15c9b0 0x7fdb9c14d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.317+0000 7fdb9bfff700 1 -- 172.21.15.53:0/2841367377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900094a0 con 0x7fdb9c161340 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdb9bfff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400ca70 tx=0x7fdb9400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:03.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400a9e0 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9400ab40 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94012400 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c0b5130 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.318+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0d15e0 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.319+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c0b52c0 con 0x7fdb9c161340 2023-11-28T11:18:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.320+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb94019030 con 0x7fdb9c161340 2023-11-28T11:18:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.321+0000 7fdb83fff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 0x7fdb8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.321+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb940856c0 con 0x7fdb9c161340 2023-11-28T11:18:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.321+0000 7fdb9b7fe700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 0x7fdb8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.322+0000 7fdb9b7fe700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 0x7fdb8806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009be0 tx=0x7fdb8c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:03.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.325+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb9404b5e0 con 0x7fdb9c161340 2023-11-28T11:18:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.573+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb9c0745c0 con 0x7fdb9c161340 2023-11-28T11:18:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.573+0000 7fdb83fff700 1 -- 172.21.15.53:0/2841367377 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7fdb94051ee0 con 0x7fdb9c161340 2023-11-28T11:18:03.574 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:03.575 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 msgr2=0x7fdb8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 0x7fdb8806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009be0 tx=0x7fdb8c009210 comp rx=0 tx=0).stop 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 msgr2=0x7fdb9c15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400ca70 tx=0x7fdb9400cd80 comp rx=0 tx=0).stop 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 shutdown_connections 2023-11-28T11:18:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdb880682f0 0x7fdb8806a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9c0d2320 0x7fdb9c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb9c161340 0x7fdb9c15ec60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 --2- 172.21.15.53:0/2841367377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb9c15c9b0 0x7fdb9c14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 >> 172.21.15.53:0/2841367377 conn(0x7fdb9c072600 msgr2=0x7fdb9c0b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 shutdown_connections 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:03.576+0000 7fdba21ff700 1 -- 172.21.15.53:0/2841367377 wait complete. 2023-11-28T11:18:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:18:03.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:17:49.188727+0000', 'modified': '2023-11-28T11:17:52.679701+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54179': {'gid': 54179, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/2501355604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 2501355604}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2501355604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:18:03.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:18:03.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:18:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.202+0000 7febf69ff700 1 Processor -- start 2023-11-28T11:18:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.202+0000 7febf69ff700 1 -- start start 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf69ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febf00d1330 0x7febf00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf69ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf69ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00b0230 con 0x7febf00d01b0 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf69ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf015cec0 con 0x7febf00d1330 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf69ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00b4570 con 0x7febf00b20b0 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febeffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7febf00d01b0 0x7febf00d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55834/0 (socket says 172.21.15.53:55834) 2023-11-28T11:18:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febeffff700 1 -- 172.21.15.53:0/1883861966 learned_addr learned my addr 172.21.15.53:0/1883861966 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febef7fe700 1 --2- 172.21.15.53:0/1883861966 >> v2:172.21.15.157:3300/0 conn(0x7febf00d1330 0x7febf00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.203+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febeeffd700 1 -- 172.21.15.53:0/1883861966 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819218645 0 0) 0x7febf00b0230 con 0x7febf00d01b0 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febeeffd700 1 -- 172.21.15.53:0/1883861966 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd80035b0 con 0x7febf00d01b0 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febef7fe700 1 -- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 msgr2=0x7febf00b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febef7fe700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febef7fe700 1 -- 172.21.15.53:0/1883861966 >> v1:172.21.15.53:6789/0 conn(0x7febf00d01b0 legacy=0x7febf00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febef7fe700 1 -- 172.21.15.53:0/1883861966 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00b4850 con 0x7febf00d1330 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.204+0000 7febef7fe700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d1330 0x7febf00b06c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febe4009840 tx=0x7febe4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a244cb9e1dc848 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febeeffd700 1 -- 172.21.15.53:0/1883861966 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011830 con 0x7febf00d1330 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febeeffd700 1 -- 172.21.15.53:0/1883861966 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4011990 con 0x7febf00d1330 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febeeffd700 1 -- 172.21.15.53:0/1883861966 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011d00 con 0x7febf00d1330 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 -- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d1330 msgr2=0x7febf00b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d1330 0x7febf00b06c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febe4009840 tx=0x7febe4009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 -- 172.21.15.53:0/1883861966 shutdown_connections 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 --2- 172.21.15.53:0/1883861966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d1330 0x7febf00b06c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 -- 172.21.15.53:0/1883861966 >> 172.21.15.53:0/1883861966 conn(0x7febf009c3f0 msgr2=0x7febf00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 -- 172.21.15.53:0/1883861966 shutdown_connections 2023-11-28T11:18:04.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.205+0000 7febf69ff700 1 -- 172.21.15.53:0/1883861966 wait complete. 2023-11-28T11:18:04.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.207+0000 7febf69ff700 1 Processor -- start 2023-11-28T11:18:04.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.207+0000 7febf69ff700 1 -- start start 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d01b0 0x7febf00ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00cc950 con 0x7febf00d1330 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00b8d80 con 0x7febf00d01b0 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf69ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf0150390 con 0x7febf00b20b0 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf4f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf4f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45720/0 (socket says 172.21.15.53:45720) 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febf4f9c700 1 -- 172.21.15.53:0/1076236082 learned_addr learned my addr 172.21.15.53:0/1076236082 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febef7fe700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d01b0 0x7febf00ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.208+0000 7febeffff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 -- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 msgr2=0x7febf00c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 -- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d01b0 msgr2=0x7febf00ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d01b0 0x7febf00ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 -- 172.21.15.53:0/1076236082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009a70 con 0x7febf00d1330 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe800efb0 tx=0x7febe800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febecff9700 1 -- 172.21.15.53:0/1076236082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800cc90 con 0x7febf00d1330 2023-11-28T11:18:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febecff9700 1 -- 172.21.15.53:0/1076236082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800f040 con 0x7febf00d1330 2023-11-28T11:18:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febecff9700 1 -- 172.21.15.53:0/1076236082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8014480 con 0x7febf00d1330 2023-11-28T11:18:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.209+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe40094a0 con 0x7febf00d1330 2023-11-28T11:18:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.210+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00afec0 con 0x7febf00d1330 2023-11-28T11:18:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.211+0000 7febecff9700 1 -- 172.21.15.53:0/1076236082 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7febe80145e0 con 0x7febf00d1330 2023-11-28T11:18:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.212+0000 7febecff9700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 0x7febdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.212+0000 7febecff9700 1 -- 172.21.15.53:0/1076236082 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febe8086b50 con 0x7febf00d1330 2023-11-28T11:18:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.213+0000 7febeffff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 0x7febdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.214+0000 7febeffff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 0x7febdc06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febe0007fd0 tx=0x7febe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.215+0000 7fec0a2dab80 1 Processor -- start 2023-11-28T11:18:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.215+0000 7fec0a2dab80 1 -- start start 2023-11-28T11:18:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 0x55997a1dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 0x55997a1f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a09bbc0 con 0x55997a1db810 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a10a350 con 0x55997a1d9130 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997a103880 con 0x55997a1dc120 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45724/0 (socket says 172.21.15.53:45724) 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 -- 172.21.15.53:0/3710449743 learned_addr learned my addr 172.21.15.53:0/3710449743 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febef7fe700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 0x55997a1dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febf4f9c700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 0x55997a1f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 -- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 msgr2=0x55997a1f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 0x55997a1f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 -- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 msgr2=0x55997a1dbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 0x55997a1dbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 -- 172.21.15.53:0/3710449743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559979f855e0 con 0x55997a1db810 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febef7fe700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 0x55997a1dbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febf4f9c700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 0x55997a1f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.216+0000 7febeffff700 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febe000f810 tx=0x7febe0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa59dd413a5d783 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7febd67fc700 1 -- 172.21.15.53:0/3710449743 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0014450 con 0x55997a1db810 2023-11-28T11:18:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7febd67fc700 1 -- 172.21.15.53:0/3710449743 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe00145b0 con 0x55997a1db810 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7febd67fc700 1 -- 172.21.15.53:0/3710449743 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0014940 con 0x55997a1db810 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3710449743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 msgr2=0x55997a1b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febe000f810 tx=0x7febe0000ed0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3710449743 shutdown_connections 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1dc120 0x55997a1f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1db810 0x55997a1b1a20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3710449743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1d9130 0x55997a1dbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.217+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3710449743 >> 172.21.15.53:0/3710449743 conn(0x55997a1aed60 msgr2=0x55997a1af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.218+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3710449743 shutdown_connections 2023-11-28T11:18:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.218+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3710449743 wait complete. 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.218+0000 7fec0a2dab80 1 -- start start 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.218+0000 7fec0a2dab80 1 Processor -- start 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979f878d0 con 0x55997a1d9130 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1cbb30 con 0x55997a1dc120 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.219+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997a1da900 con 0x55997a1db810 2023-11-28T11:18:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7fec0a2dab80 10 client.0 _ll_register_callbacks cb 0x559979fa9240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45734/0 (socket says 172.21.15.53:45734) 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 -- 172.21.15.53:0/3299611982 learned_addr learned my addr 172.21.15.53:0/3299611982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febef7fe700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febf4f9c700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 -- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 msgr2=0x55997a1dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 -- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 msgr2=0x55997a1dd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1dd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 -- 172.21.15.53:0/3299611982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd80035b0 con 0x55997a1d9130 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febf4f9c700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1dd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.220+0000 7febeffff700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febe0006920 tx=0x7febe0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18efd3fbeecd9613 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0004a70 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55997a0d1140 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febe0004bd0 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe0004d90 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 10 client.4599 fetch_fsmap learned FSMap version 8 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x559979f31d90 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997a0d1140 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7febe0032060 con 0x55997a1d9130 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 10 client.4599 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 10 client.4599 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.221+0000 7fec0a2dab80 20 client.4599 populate_metadata read hostname 'smithi053' 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a1cbd30 0x55997a1cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7fec0a2dab80 4 client.4599 mds_command: new command op to 54179 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:18:04.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 --> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559979f87f10 con 0x55997a1cbd30 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7febef7fe700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a1cbd30 0x55997a1cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7febef7fe700 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a1cbd30 0x55997a1cae40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a463140eb7e9ed80 server_cookie=f3d0caa1520f036a in_seq=0 out_seq=0 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.222+0000 7febcf7fe700 10 client.4599 ms_handle_connect on v2:172.21.15.53:6833/2501355604 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7febcf7fe700 1 -- 172.21.15.53:0/3299611982 <== mds.0 v2:172.21.15.53:6833/2501355604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x559979f87f10 con 0x55997a1cbd30 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7febcf7fe700 10 client.4599 handle_command_reply: tid=0 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 client.4599 shutdown 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 msgr2=0x55997a1ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febe0006920 tx=0x7febe0000bc0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 shutdown_connections 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a1cbd30 0x55997a1cae40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1dd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a1ca140 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/3299611982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7febcf7fe700 0 client.4599 ms_handle_reset on v2:172.21.15.53:6833/2501355604 2023-11-28T11:18:04.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 >> 172.21.15.53:0/3299611982 conn(0x55997a1aed60 msgr2=0x55997a1b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.223+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 shutdown_connections 2023-11-28T11:18:04.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.224+0000 7fec0a2dab80 1 -- 172.21.15.53:0/3299611982 wait complete. 2023-11-28T11:18:04.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.224+0000 7fec0a2dab80 20 client.4599 trim_cache size 0 max 16384 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.241+0000 7fec0a2dab80 1 Processor -- start 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 -- start start 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1d9130 0x55997a255080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1db810 0x55997a1c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1aaef0 con 0x55997a1dc120 2023-11-28T11:18:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1d1e20 con 0x55997a1db810 2023-11-28T11:18:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.242+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997a20f4a0 con 0x55997a1d9130 2023-11-28T11:18:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45740/0 (socket says 172.21.15.53:45740) 2023-11-28T11:18:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 -- 172.21.15.53:0/427838166 learned_addr learned my addr 172.21.15.53:0/427838166 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 -- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1d9130 msgr2=0x55997a255080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1d9130 0x55997a255080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 -- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1db810 msgr2=0x55997a1c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febef7fe700 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1db810 0x55997a1c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1db810 0x55997a1c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 -- 172.21.15.53:0/427838166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997a0ca140 con 0x55997a1dc120 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febf4f9c700 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7febe8053480 tx=0x7febe8056a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9172d93150597d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febd6ffd700 1 -- 172.21.15.53:0/427838166 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8056670 con 0x55997a1dc120 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febd6ffd700 1 -- 172.21.15.53:0/427838166 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe805eb80 con 0x55997a1dc120 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.243+0000 7febd6ffd700 1 -- 172.21.15.53:0/427838166 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8024bd0 con 0x55997a1dc120 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 -- 172.21.15.53:0/427838166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 msgr2=0x55997a1ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7febe8053480 tx=0x7febe8056a10 comp rx=0 tx=0).stop 2023-11-28T11:18:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 -- 172.21.15.53:0/427838166 shutdown_connections 2023-11-28T11:18:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1d9130 0x55997a255080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1dc120 0x55997a1ca490 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/427838166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1db810 0x55997a1c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 -- 172.21.15.53:0/427838166 >> 172.21.15.53:0/427838166 conn(0x55997a1aed60 msgr2=0x55997a1de4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 -- 172.21.15.53:0/427838166 shutdown_connections 2023-11-28T11:18:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.244+0000 7fec0a2dab80 1 -- 172.21.15.53:0/427838166 wait complete. 2023-11-28T11:18:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.245+0000 7fec0a2dab80 1 -- start start 2023-11-28T11:18:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.245+0000 7fec0a2dab80 1 Processor -- start 2023-11-28T11:18:04.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1b0bd0 con 0x55997a1d9130 2023-11-28T11:18:04.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1d9b60 con 0x55997a1dc120 2023-11-28T11:18:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997a254930 con 0x55997a1db810 2023-11-28T11:18:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7fec0a2dab80 10 client.0 _ll_register_callbacks cb 0x559979fa9240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:18:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45752/0 (socket says 172.21.15.53:45752) 2023-11-28T11:18:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 -- 172.21.15.53:0/1528383200 learned_addr learned my addr 172.21.15.53:0/1528383200 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febf4f9c700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febef7fe700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 -- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 msgr2=0x55997a1b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 -- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 msgr2=0x55997a1b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febeffff700 1 -- 172.21.15.53:0/1528383200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00afec0 con 0x55997a1d9130 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.247+0000 7febef7fe700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febeffff700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febe000f7e0 tx=0x7febe0007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51436e7dff09c460 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0004c80 con 0x55997a1d9130 2023-11-28T11:18:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febe003dc50 con 0x55997a1d9130 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55997a0c1160 con 0x55997a1d9130 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe000fbd0 con 0x55997a1d9130 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 10 client.4605 fetch_fsmap learned FSMap version 8 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x559979f58200 con 0x55997a1d9130 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997a0c1160 con 0x55997a1d9130 2023-11-28T11:18:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7febe0032070 con 0x55997a1d9130 2023-11-28T11:18:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 10 client.4605 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:18:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 10 client.4605 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:18:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 20 client.4605 populate_metadata read hostname 'smithi053' 2023-11-28T11:18:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a218ad0 0x55997a1d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 4 client.4605 mds_command: new command op to 54179 tid=0[{"prefix": "status"}] 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.248+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 --> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55997a0ab170 con 0x55997a218ad0 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7febef7fe700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a218ad0 0x55997a1d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7febef7fe700 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a218ad0 0x55997a1d2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c86728c2b83649 server_cookie=69bf714a0851521b in_seq=0 out_seq=0 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7febcf7fe700 10 client.4605 ms_handle_connect on v2:172.21.15.53:6833/2501355604 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7febcf7fe700 1 -- 172.21.15.53:0/1528383200 <== mds.0 v2:172.21.15.53:6833/2501355604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55997a0ab170 con 0x55997a218ad0 2023-11-28T11:18:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7febcf7fe700 10 client.4605 handle_command_reply: tid=0 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 client.4605 shutdown 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 msgr2=0x55997a2543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febe000f7e0 tx=0x7febe0007da0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 shutdown_connections 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:6833/2501355604,v1:172.21.15.53:6835/2501355604] conn(0x55997a218ad0 0x55997a1d2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997a1dc120 0x55997a1b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997a1d9130 0x55997a2543f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 --2- 172.21.15.53:0/1528383200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55997a1db810 0x55997a1b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.249+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 >> 172.21.15.53:0/1528383200 conn(0x55997a1aed60 msgr2=0x55997a1de4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febcf7fe700 0 client.4605 ms_handle_reset on v2:172.21.15.53:6833/2501355604 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 shutdown_connections 2023-11-28T11:18:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7fec0a2dab80 1 -- 172.21.15.53:0/1528383200 wait complete. 2023-11-28T11:18:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7fec0a2dab80 20 client.4605 trim_cache size 0 max 16384 2023-11-28T11:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "38901cf1-eb03-4cf6-8c7d-44ad237eb202", 2023-11-28T11:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-28T11:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 54179, 2023-11-28T11:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-28T11:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-28T11:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 15.058347806, 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 8, 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 22, 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 22, 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 20.605759474999999 2023-11-28T11:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 msgr2=0x7febdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 0x7febdc06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febe0007fd0 tx=0x7febe0009040 comp rx=0 tx=0).stop 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 msgr2=0x7febf00ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe800efb0 tx=0x7febe800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 shutdown_connections 2023-11-28T11:18:04.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7febdc0682f0 0x7febdc06a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febf00b20b0 0x7febf00c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf00d1330 0x7febf00ca660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 --2- 172.21.15.53:0/1076236082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf00d01b0 0x7febf00ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:04.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.250+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 >> 172.21.15.53:0/1076236082 conn(0x7febf009c3f0 msgr2=0x7febf00b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:04.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.251+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 shutdown_connections 2023-11-28T11:18:04.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:04.251+0000 7febf69ff700 1 -- 172.21.15.53:0/1076236082 wait complete. 2023-11-28T11:18:04.280 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T11:18:04.312 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T11:18:04.312 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:18:04.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.313 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:18:04.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-28T11:18:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-11-28T11:18:04.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:18:04.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.474 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:18:04.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-28T11:18:04.628 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-11-28T11:18:04.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:18:04.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.632 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:18:04.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-28T11:18:04.759 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-11-28T11:18:04.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:18:04.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.762 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:18:04.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:04.806 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-28T11:18:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-11-28T11:18:04.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:18:04.921 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T11:18:04.921 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:18:04.922 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:18:04.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:18:05.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.123+0000 7fce823ff700 1 Processor -- start 2023-11-28T11:18:05.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.123+0000 7fce823ff700 1 -- start start 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.124+0000 7fce823ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c15c9b0 0x7fce7c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.124+0000 7fce823ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce823ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0ce160 con 0x7fce7c161340 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce823ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0b5800 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce823ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0d1950 con 0x7fce7c15c9b0 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce7bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c15c9b0 0x7fce7c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:33358/0 (socket says 172.21.15.53:33358) 2023-11-28T11:18:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 -- 172.21.15.53:0/2776081379 learned_addr learned my addr 172.21.15.53:0/2776081379 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 -- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c15c9b0 msgr2=0x7fce7c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 --2- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c15c9b0 0x7fce7c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 -- 172.21.15.53:0/2776081379 >> v1:172.21.15.157:6789/0 conn(0x7fce7c161340 legacy=0x7fce7c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.125+0000 7fce8099c700 1 -- 172.21.15.53:0/2776081379 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0d24f0 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce8099c700 1 --2- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e50 tx=0x7fce7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcca5ad81f173e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce7affd700 1 -- 172.21.15.53:0/2776081379 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400e070 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce7affd700 1 -- 172.21.15.53:0/2776081379 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400b910 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce7affd700 1 -- 172.21.15.53:0/2776081379 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400bcc0 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce823ff700 1 -- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 msgr2=0x7fce7c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.126+0000 7fce823ff700 1 --2- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e50 tx=0x7fce7400b330 comp rx=0 tx=0).stop 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 -- 172.21.15.53:0/2776081379 shutdown_connections 2023-11-28T11:18:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 --2- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c0cdc20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 --2- 172.21.15.53:0/2776081379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c15c9b0 0x7fce7c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 -- 172.21.15.53:0/2776081379 >> 172.21.15.53:0/2776081379 conn(0x7fce7c072600 msgr2=0x7fce7c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 -- 172.21.15.53:0/2776081379 shutdown_connections 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.127+0000 7fce823ff700 1 -- 172.21.15.53:0/2776081379 wait complete. 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.128+0000 7fce823ff700 1 Processor -- start 2023-11-28T11:18:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.129+0000 7fce823ff700 1 -- start start 2023-11-28T11:18:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.129+0000 7fce823ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce823ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce823ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 0x7fce7c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce823ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c158a20 con 0x7fce7c15c9b0 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce823ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c158d00 con 0x7fce7c0cd7c0 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce823ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c158fe0 con 0x7fce7c161340 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45766/0 (socket says 172.21.15.53:45766) 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 -- 172.21.15.53:0/1844811346 learned_addr learned my addr 172.21.15.53:0/1844811346 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce8099c700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 0x7fce7c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7bfff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 -- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 msgr2=0x7fce7c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 0x7fce7c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 -- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 msgr2=0x7fce7c147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 -- 172.21.15.53:0/1844811346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009b30 con 0x7fce7c15c9b0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce8099c700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 0x7fce7c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.130+0000 7fce7b7fe700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c450 tx=0x7fce7000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.131+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700052d0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.131+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70005430 con 0x7fce7c15c9b0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.131+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700038c0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.131+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0d1300 con 0x7fce7c15c9b0 2023-11-28T11:18:05.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.131+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c0b5520 con 0x7fce7c15c9b0 2023-11-28T11:18:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.132+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c0d15e0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.133+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce70019030 con 0x7fce7c15c9b0 2023-11-28T11:18:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.134+0000 7fce78ff9700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 0x7fce6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.134+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce70084f70 con 0x7fce7c15c9b0 2023-11-28T11:18:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.134+0000 7fce7bfff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 0x7fce6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.135+0000 7fce7bfff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 0x7fce6806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009ad0 tx=0x7fce6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:05.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.137+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce700516e0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.389+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce7c1597e0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.389+0000 7fce78ff9700 1 -- 172.21.15.53:0/1844811346 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce7c1597e0 con 0x7fce7c15c9b0 2023-11-28T11:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 msgr2=0x7fce6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 0x7fce6806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009ad0 tx=0x7fce6c008040 comp rx=0 tx=0).stop 2023-11-28T11:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 msgr2=0x7fce7c154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c450 tx=0x7fce7000c810 comp rx=0 tx=0).stop 2023-11-28T11:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 shutdown_connections 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce680682f0 0x7fce6806a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7c161340 0x7fce7c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0cd7c0 0x7fce7c147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 --2- 172.21.15.53:0/1844811346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c15c9b0 0x7fce7c154e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 >> 172.21.15.53:0/1844811346 conn(0x7fce7c072600 msgr2=0x7fce7c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 shutdown_connections 2023-11-28T11:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:05.392+0000 7fce823ff700 1 -- 172.21.15.53:0/1844811346 wait complete. 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:18:05.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:18:05.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:05.412 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-11-28T11:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42133sec preferred_lft 42133sec 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-11-28T11:18:05.457 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:18:05.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:18:05.458 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:05.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:05.944 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:18:06.016 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:18:06.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:06.019 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-28T11:18:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-28T11:18:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:18:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:18:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:18:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:18:06.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:18:06.081 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:06.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:06.782 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:18:06.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:06.826 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-11-28T11:18:06.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:06.911 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:06.911 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:06.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:18:06.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:18:06.911 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:07.095 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:18:07.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:07.095 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:07.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:07.375 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:07.376 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:07.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-11-28T11:18:07.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:18:07.376 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:07.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:07.551 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:18:07.594 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:18:07.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:07.595 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:18:07.678 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:18:07.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:07.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:18:07.840 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:18:07.841 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:18:07.841 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:18:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:18:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:08.274 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:08.274 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:08.300 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:18:08.300 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:18:08.300 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:08.300 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:08.300 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:08.301 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:18:08.301 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:18:08.301 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:18:08.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:18:08.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:08.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:18:08.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:08.377 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:08.487 INFO:teuthology.orchestra.run.smithi053.stdout:client4611 2023-11-28T11:18:08.488 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:08.488 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:08.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:08.531 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:08.665 INFO:teuthology.orchestra.run.smithi053.stdout:client4611 2023-11-28T11:18:08.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:08.665 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:08.750 INFO:teuthology.orchestra.run.smithi053.stdout:client4611 2023-11-28T11:18:08.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:08.751 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38901cf1-eb03-4cf6-8c7d-44ad237eb202.client4611/status') 2023-11-28T11:18:08.858 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.4611 (3)192.168.0.1:0/2712612171 2023-11-28T11:18:08.858 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2023-11-28T11:18:08.859 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2023-11-28T11:18:08.859 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2023-11-28T11:18:08.859 INFO:teuthology.orchestra.run.smithi053.stderr:69 bytes copied, 5.6567e-05 s, 1.2 MB/s 2023-11-28T11:18:08.861 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T11:18:08.862 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:18:08.862 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:18:08.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:18:09.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.061+0000 7f400e9ff700 1 Processor -- start 2023-11-28T11:18:09.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.062+0000 7f400e9ff700 1 -- start start 2023-11-28T11:18:09.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400e9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 0x7f40080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400e9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40080d2320 0x7f40080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400e9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080d1950 con 0x7f400815c9b0 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400e9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080b5800 con 0x7f40080d2320 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400e9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080b5300 con 0x7f4008161340 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f4007fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f400815c9b0 0x7f40080b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55850/0 (socket says 172.21.15.53:55850) 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f4007fff700 1 -- 172.21.15.53:0/3211046451 learned_addr learned my addr 172.21.15.53:0/3211046451 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f40077fe700 1 --2- 172.21.15.53:0/3211046451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 0x7f40080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f400cf9c700 1 --2- 172.21.15.53:0/3211046451 >> v2:172.21.15.157:3300/0 conn(0x7f40080d2320 0x7f40080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f4006ffd700 1 -- 172.21.15.53:0/3211046451 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410139939 0 0) 0x7f40080d1950 con 0x7f400815c9b0 2023-11-28T11:18:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.063+0000 7f4006ffd700 1 -- 172.21.15.53:0/3211046451 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff00035b0 con 0x7f400815c9b0 2023-11-28T11:18:09.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f40077fe700 1 -- 172.21.15.53:0/3211046451 >> v2:172.21.15.157:3300/0 conn(0x7f40080d2320 msgr2=0x7f40080cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:09.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f40077fe700 1 --2- 172.21.15.53:0/3211046451 >> v2:172.21.15.157:3300/0 conn(0x7f40080d2320 0x7f40080cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f40077fe700 1 -- 172.21.15.53:0/3211046451 >> v1:172.21.15.53:6789/0 conn(0x7f400815c9b0 legacy=0x7f40080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f40077fe700 1 -- 172.21.15.53:0/3211046451 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080b5520 con 0x7f4008161340 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f40077fe700 1 --2- 172.21.15.53:0/3211046451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 0x7f40080d1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009840 tx=0x7f3ffc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99867dd23372a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f4006ffd700 1 -- 172.21.15.53:0/3211046451 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc011830 con 0x7f4008161340 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f4006ffd700 1 -- 172.21.15.53:0/3211046451 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc011990 con 0x7f4008161340 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.064+0000 7f4006ffd700 1 -- 172.21.15.53:0/3211046451 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc011d20 con 0x7f4008161340 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 -- 172.21.15.53:0/3211046451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 msgr2=0x7f40080d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 --2- 172.21.15.53:0/3211046451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 0x7f40080d1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009840 tx=0x7f3ffc009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 -- 172.21.15.53:0/3211046451 shutdown_connections 2023-11-28T11:18:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 --2- 172.21.15.53:0/3211046451 >> v2:172.21.15.157:3300/0 conn(0x7f40080d2320 0x7f40080cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 --2- 172.21.15.53:0/3211046451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4008161340 0x7f40080d1de0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 -- 172.21.15.53:0/3211046451 >> 172.21.15.53:0/3211046451 conn(0x7f4008072600 msgr2=0x7f40080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 -- 172.21.15.53:0/3211046451 shutdown_connections 2023-11-28T11:18:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.065+0000 7f400e9ff700 1 -- 172.21.15.53:0/3211046451 wait complete. 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.067+0000 7f400e9ff700 1 Processor -- start 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.067+0000 7f400e9ff700 1 -- start start 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40080d2320 0x7f40081508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400815c9b0 0x7f4008152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008157980 con 0x7f4008161340 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008150e20 con 0x7f400815c9b0 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400e9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40081536d0 con 0x7f40080d2320 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400cf9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400cf9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45782/0 (socket says 172.21.15.53:45782) 2023-11-28T11:18:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f400cf9c700 1 -- 172.21.15.53:0/4036011762 learned_addr learned my addr 172.21.15.53:0/4036011762 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f4007fff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40080d2320 0x7f40081508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.068+0000 7f40077fe700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400815c9b0 0x7f4008152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 -- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40080d2320 msgr2=0x7f40081508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40080d2320 0x7f40081508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 -- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400815c9b0 msgr2=0x7f4008152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400815c9b0 0x7f4008152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 -- 172.21.15.53:0/4036011762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0094a0 con 0x7f4008161340 2023-11-28T11:18:09.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400cf9c700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f400000efc0 tx=0x7f400000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000cad0 con 0x7f4008161340 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000f040 con 0x7f4008161340 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40080d15e0 con 0x7f4008161340 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000ce70 con 0x7f4008161340 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.069+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40080d1300 con 0x7f4008161340 2023-11-28T11:18:09.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.070+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400815cd80 con 0x7f4008161340 2023-11-28T11:18:09.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.072+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4000014480 con 0x7f4008161340 2023-11-28T11:18:09.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.073+0000 7f4004ff9700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 0x7f3ff406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:09.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.073+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4000086520 con 0x7f4008161340 2023-11-28T11:18:09.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.073+0000 7f4007fff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 0x7f3ff406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:09.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.074+0000 7f4007fff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 0x7f3ff406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8007fd0 tx=0x7f3ff8007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:09.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.077+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4000052d20 con 0x7f4008161340 2023-11-28T11:18:09.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.326+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f40080745c0 con 0x7f4008161340 2023-11-28T11:18:09.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.326+0000 7f4004ff9700 1 -- 172.21.15.53:0/4036011762 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4000056340 con 0x7f4008161340 2023-11-28T11:18:09.329 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.328+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 msgr2=0x7f3ff406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.328+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 0x7f3ff406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8007fd0 tx=0x7f3ff8007dc0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.328+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 msgr2=0x7f4008153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.328+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f400000efc0 tx=0x7f400000c0c0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 shutdown_connections 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3ff40682f0 0x7f3ff406a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40080d2320 0x7f40081508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008161340 0x7f4008153070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 --2- 172.21.15.53:0/4036011762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400815c9b0 0x7f4008152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:09.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 >> 172.21.15.53:0/4036011762 conn(0x7f4008072600 msgr2=0x7f400815f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:09.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 shutdown_connections 2023-11-28T11:18:09.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:09.329+0000 7f400e9ff700 1 -- 172.21.15.53:0/4036011762 wait complete. 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:18:09.349 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:18:09.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:18:09.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:18:09.350 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:18:09.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.350 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.395 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:09.396 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42129sec preferred_lft 42129sec 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:09.397 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::892:d6ff:fe9c:c696/64 scope link 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::86e:80ff:fe63:4cd1/64 scope link 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:09.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.399 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:18:09.502 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:18:09.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.506 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-28T11:18:09.552 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-28T11:18:09.552 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:18:09.552 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:18:09.552 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:18:09.552 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:18:09.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.553 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:09.553 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:09.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:18:09.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:18:09.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:18:09.554 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:09.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.728 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:18:09.772 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:18:09.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.772 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-11-28T11:18:09.855 INFO:teuthology.orchestra.run.smithi053.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:18:09.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:09.857 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:09.857 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:09.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:18:09.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:18:09.857 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:10.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:18:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 8a:f1:87:28:b2:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::88f1:87ff:fe28:b203/64 scope link 2023-11-28T11:18:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:18:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 8a:f1:87:28:b2:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::88f1:87ff:fe28:b203/64 scope link 2023-11-28T11:18:10.184 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:10.185 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:18:10.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:18:10.186 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:10.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.436 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:18:10.436 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:18:10.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.1 up 2023-11-28T11:18:10.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:18:10.436 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:18:10.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:10.610 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:18:10.654 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:18:10.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:10.655 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:18:10.737 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:18:10.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:18:10.884 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:18:10.884 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:18:10.884 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:10.903 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:18:10.903 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:18:10.903 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:10.903 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:10.903 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:18:10.926 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:18:10.931 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:18:10.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:10.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:18:11.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.001 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:11.086 INFO:teuthology.orchestra.run.smithi053.stdout:client54449 2023-11-28T11:18:11.087 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:11.087 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:11.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.130 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:11.255 INFO:teuthology.orchestra.run.smithi053.stdout:client54449 2023-11-28T11:18:11.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.256 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:11.341 INFO:teuthology.orchestra.run.smithi053.stdout:client54449 2023-11-28T11:18:11.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.342 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38901cf1-eb03-4cf6-8c7d-44ad237eb202.client54449/status') 2023-11-28T11:18:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.54449 (3)172.21.15.53:0/2986752968 2023-11-28T11:18:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2023-11-28T11:18:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2023-11-28T11:18:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2023-11-28T11:18:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:71 bytes copied, 7.1486e-05 s, 993 kB/s 2023-11-28T11:18:11.451 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T11:18:11.451 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:18:11.452 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:18:11.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:18:11.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.651+0000 7f06801ff700 1 Processor -- start 2023-11-28T11:18:11.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.652+0000 7f06801ff700 1 -- start start 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f06801ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f067815c9b0 0x7f06780b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f06801ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0678161340 0x7f06780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f06801ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780d1950 con 0x7f06780d2320 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f06801ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780b5800 con 0x7f0678161340 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f06801ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780b5520 con 0x7f067815c9b0 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0678161340 0x7f06780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0678161340 0x7f06780d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53980/0 (socket says 172.21.15.53:53980) 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 -- 172.21.15.53:0/3273127917 learned_addr learned my addr 172.21.15.53:0/3273127917 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067df9b700 1 --2- 172.21.15.53:0/3273127917 >> v2:172.21.15.157:3300/0 conn(0x7f067815c9b0 0x7f06780b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 -- 172.21.15.53:0/3273127917 >> v2:172.21.15.157:3300/0 conn(0x7f067815c9b0 msgr2=0x7f06780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 --2- 172.21.15.53:0/3273127917 >> v2:172.21.15.157:3300/0 conn(0x7f067815c9b0 0x7f06780b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 -- 172.21.15.53:0/3273127917 >> v1:172.21.15.157:6789/0 conn(0x7f06780d2320 legacy=0x7f06780cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.653+0000 7f067d79a700 1 -- 172.21.15.53:0/3273127917 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780d1300 con 0x7f0678161340 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.654+0000 7f067d79a700 1 --2- 172.21.15.53:0/3273127917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f06780d1de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0678067460 tx=0x7f0668009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231cf447775fda7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.654+0000 7f067cf99700 1 -- 172.21.15.53:0/3273127917 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668011830 con 0x7f0678161340 2023-11-28T11:18:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.654+0000 7f067cf99700 1 -- 172.21.15.53:0/3273127917 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0668011990 con 0x7f0678161340 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.654+0000 7f067cf99700 1 -- 172.21.15.53:0/3273127917 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668011d20 con 0x7f0678161340 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 -- 172.21.15.53:0/3273127917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 msgr2=0x7f06780d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 --2- 172.21.15.53:0/3273127917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f06780d1de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0678067460 tx=0x7f0668009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 -- 172.21.15.53:0/3273127917 shutdown_connections 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 --2- 172.21.15.53:0/3273127917 >> v2:172.21.15.157:3300/0 conn(0x7f067815c9b0 0x7f06780b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 --2- 172.21.15.53:0/3273127917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f06780d1de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 -- 172.21.15.53:0/3273127917 >> 172.21.15.53:0/3273127917 conn(0x7f0678072600 msgr2=0x7f06780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 -- 172.21.15.53:0/3273127917 shutdown_connections 2023-11-28T11:18:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.655+0000 7f06801ff700 1 -- 172.21.15.53:0/3273127917 wait complete. 2023-11-28T11:18:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.657+0000 7f06801ff700 1 Processor -- start 2023-11-28T11:18:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.657+0000 7f06801ff700 1 -- start start 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06780d2320 0x7f06780c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 0x7f06781f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780c65a0 con 0x7f0678161340 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678154970 con 0x7f067815c9b0 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f06801ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06780c8830 con 0x7f06780d2320 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53990/0 (socket says 172.21.15.53:53990) 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 -- 172.21.15.53:0/2240957200 learned_addr learned my addr 172.21.15.53:0/2240957200 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067d79a700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 0x7f06781f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067df9b700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06780d2320 0x7f06780c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 -- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06780d2320 msgr2=0x7f06780c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06780d2320 0x7f06780c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 -- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 msgr2=0x7f06781f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 0x7f06781f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067e79c700 1 -- 172.21.15.53:0/2240957200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0668009550 con 0x7f0678161340 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.659+0000 7f067d79a700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 0x7f06781f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f067e79c700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f067000cbc0 tx=0x7f067000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670003c00 con 0x7f0678161340 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0670003d60 con 0x7f0678161340 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000ab90 con 0x7f0678161340 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06780b5130 con 0x7f0678161340 2023-11-28T11:18:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.660+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06780d15e0 con 0x7f0678161340 2023-11-28T11:18:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.661+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06780b52c0 con 0x7f0678161340 2023-11-28T11:18:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.662+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0670010050 con 0x7f0678161340 2023-11-28T11:18:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.663+0000 7f066e7fc700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 0x7f065c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.663+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0670085bd0 con 0x7f0678161340 2023-11-28T11:18:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.663+0000 7f067df9b700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 0x7f065c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:11.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.664+0000 7f067df9b700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 0x7f065c06a7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0674009b70 tx=0x7f0674008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.667+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06700523c0 con 0x7f0678161340 2023-11-28T11:18:11.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.916+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06780745c0 con 0x7f0678161340 2023-11-28T11:18:11.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.916+0000 7f066e7fc700 1 -- 172.21.15.53:0/2240957200 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06700559e0 con 0x7f0678161340 2023-11-28T11:18:11.918 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:18:11.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 msgr2=0x7f065c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 0x7f065c06a7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0674009b70 tx=0x7f0674008040 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 msgr2=0x7f067806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f067000cbc0 tx=0x7f067000cf80 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 shutdown_connections 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f065c068340 0x7f065c06a7f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06780d2320 0x7f06780c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678161340 0x7f067806a900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 --2- 172.21.15.53:0/2240957200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067815c9b0 0x7f06781f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.918+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 >> 172.21.15.53:0/2240957200 conn(0x7f0678072600 msgr2=0x7f06780b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.919+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 shutdown_connections 2023-11-28T11:18:11.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:11.919+0000 7f06801ff700 1 -- 172.21.15.53:0/2240957200 wait complete. 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:18:11.939 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:18:11.940 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:18:11.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:18:11.940 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:18:11.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.940 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-28T11:18:11.988 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-28T11:18:11.989 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42153sec preferred_lft 42153sec 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::8530:f080:ccdf:ff7b/64 scope link noprefixroute 2023-11-28T11:18:11.990 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:11.991 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:18:11.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:11.991 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:11.991 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:11.991 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:18:11.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2023-11-28T11:18:11.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2023-11-28T11:18:11.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:18:11.992 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:12.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:12.423 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:18:12.495 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-28T11:18:12.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:12.499 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-28T11:18:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-28T11:18:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:18:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:18:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:18:12.560 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:18:12.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:12.561 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:12.561 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:12.561 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:18:12.561 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:18:12.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:18:12.562 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:13.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:13.261 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:18:13.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:13.307 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-11-28T11:18:13.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:13.391 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:13.391 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:13.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:18:13.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:18:13.391 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:13.575 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:18:13.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:18:13.576 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:13.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:13.865 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:13.865 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:13.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-28T11:18:13.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:18:13.865 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:14.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:14.039 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:18:14.083 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:18:14.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:14.083 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:18:14.166 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:18:14.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:14.166 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:18:14.321 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:18:14.322 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:18:14.322 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:14.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:18:14.370 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:18:14.827 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:18:14.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:18:14.827 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:14.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:14.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:14.852 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:18:14.853 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:18:14.857 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:18:14.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:14.858 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:18:14.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:14.928 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:15.024 INFO:teuthology.orchestra.run.smithi157.stdout:client4626 2023-11-28T11:18:15.024 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:15.024 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:15.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:15.068 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:15.194 INFO:teuthology.orchestra.run.smithi157.stdout:client4626 2023-11-28T11:18:15.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:15.194 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:15.280 INFO:teuthology.orchestra.run.smithi157.stdout:client4626 2023-11-28T11:18:15.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:15.281 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38901cf1-eb03-4cf6-8c7d-44ad237eb202.client4626/status') 2023-11-28T11:18:15.389 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4626 (3)172.21.15.157:0/500036436 2023-11-28T11:18:15.389 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2023-11-28T11:18:15.390 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2023-11-28T11:18:15.390 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2023-11-28T11:18:15.390 INFO:teuthology.orchestra.run.smithi157.stderr:70 bytes copied, 5.6767e-05 s, 1.2 MB/s 2023-11-28T11:18:15.391 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T11:18:15.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:18:15.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:18:15.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:18:15.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.602+0000 7f47641ff700 1 Processor -- start 2023-11-28T11:18:15.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.603+0000 7f47641ff700 1 -- start start 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f47641ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f475c161340 0x7f475c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f47641ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f475c0d2320 0x7f475c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f47641ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c0d1950 con 0x7f475c15c9b0 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f47641ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c0b5800 con 0x7f475c161340 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f47641ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c0b5300 con 0x7f475c0d2320 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f476179a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f475c161340 0x7f475c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f4761f9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f475c15c9b0 0x7f475c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48784/0 (socket says 172.21.15.53:48784) 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f4761f9b700 1 -- 172.21.15.53:0/158436293 learned_addr learned my addr 172.21.15.53:0/158436293 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.604+0000 7f476279c700 1 --2- 172.21.15.53:0/158436293 >> v2:172.21.15.157:3300/0 conn(0x7f475c0d2320 0x7f475c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f476179a700 1 -- 172.21.15.53:0/158436293 >> v2:172.21.15.157:3300/0 conn(0x7f475c0d2320 msgr2=0x7f475c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f476179a700 1 --2- 172.21.15.53:0/158436293 >> v2:172.21.15.157:3300/0 conn(0x7f475c0d2320 0x7f475c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f476179a700 1 -- 172.21.15.53:0/158436293 >> v1:172.21.15.53:6789/0 conn(0x7f475c15c9b0 legacy=0x7f475c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f476179a700 1 -- 172.21.15.53:0/158436293 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0b5520 con 0x7f475c161340 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f4760f99700 1 -- 172.21.15.53:0/158436293 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653910854 0 0) 0x7f475c0d1950 con 0x7f475c15c9b0 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f476179a700 1 --2- 172.21.15.53:0/158436293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c0d1de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f475c06c170 tx=0x7f474c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860bd3d84cd5197a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f4760f99700 1 -- 172.21.15.53:0/158436293 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c011830 con 0x7f475c161340 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f4760f99700 1 -- 172.21.15.53:0/158436293 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c011990 con 0x7f475c161340 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.605+0000 7f4760f99700 1 -- 172.21.15.53:0/158436293 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c011d20 con 0x7f475c161340 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 -- 172.21.15.53:0/158436293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 msgr2=0x7f475c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 --2- 172.21.15.53:0/158436293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c0d1de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f475c06c170 tx=0x7f474c009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 -- 172.21.15.53:0/158436293 shutdown_connections 2023-11-28T11:18:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 --2- 172.21.15.53:0/158436293 >> v2:172.21.15.157:3300/0 conn(0x7f475c0d2320 0x7f475c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 --2- 172.21.15.53:0/158436293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c0d1de0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 -- 172.21.15.53:0/158436293 >> 172.21.15.53:0/158436293 conn(0x7f475c072600 msgr2=0x7f475c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:15.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 -- 172.21.15.53:0/158436293 shutdown_connections 2023-11-28T11:18:15.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.606+0000 7f47641ff700 1 -- 172.21.15.53:0/158436293 wait complete. 2023-11-28T11:18:15.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.608+0000 7f47641ff700 1 Processor -- start 2023-11-28T11:18:15.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.608+0000 7f47641ff700 1 -- start start 2023-11-28T11:18:15.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f475c0d2320 0x7f475c1e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475c15c9b0 0x7f475c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1e7f00 con 0x7f475c161340 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1e6940 con 0x7f475c0d2320 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f47641ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1e6c20 con 0x7f475c15c9b0 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f476279c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f476279c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54002/0 (socket says 172.21.15.53:54002) 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f476279c700 1 -- 172.21.15.53:0/1376917517 learned_addr learned my addr 172.21.15.53:0/1376917517 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f4761f9b700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f475c0d2320 0x7f475c1e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.609+0000 7f476179a700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475c15c9b0 0x7f475c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 -- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475c15c9b0 msgr2=0x7f475c0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475c15c9b0 0x7f475c0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 -- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f475c0d2320 msgr2=0x7f475c1e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f475c0d2320 0x7f475c1e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 -- 172.21.15.53:0/1376917517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0094a0 con 0x7f475c161340 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f476279c700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4754009fd0 tx=0x7f475400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400ede0 con 0x7f475c161340 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754014420 con 0x7f475c161340 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540147d0 con 0x7f475c161340 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c0d15e0 con 0x7f475c161340 2023-11-28T11:18:15.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.610+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c0d1300 con 0x7f475c161340 2023-11-28T11:18:15.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.611+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c15cd80 con 0x7f475c161340 2023-11-28T11:18:15.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.612+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4754012050 con 0x7f475c161340 2023-11-28T11:18:15.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.613+0000 7f47527fc700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 0x7f474006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:15.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.613+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47540863d0 con 0x7f475c161340 2023-11-28T11:18:15.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.614+0000 7f4761f9b700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 0x7f474006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:15.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.614+0000 7f4761f9b700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 0x7f474006a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4758000f60 tx=0x7f4758007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:15.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.618+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4754052b40 con 0x7f475c161340 2023-11-28T11:18:15.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.868+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f475c1e7830 con 0x7f475c161340 2023-11-28T11:18:15.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.869+0000 7f47527fc700 1 -- 172.21.15.53:0/1376917517 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4754056160 con 0x7f475c161340 2023-11-28T11:18:15.871 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:18:15.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 msgr2=0x7f474006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 0x7f474006a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4758000f60 tx=0x7f4758007c90 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 msgr2=0x7f475c14a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4754009fd0 tx=0x7f475400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 shutdown_connections 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47400682f0 0x7f474006a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f475c161340 0x7f475c14a310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f475c0d2320 0x7f475c1e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 --2- 172.21.15.53:0/1376917517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475c15c9b0 0x7f475c0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 >> 172.21.15.53:0/1376917517 conn(0x7f475c072600 msgr2=0x7f475c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.871+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 shutdown_connections 2023-11-28T11:18:15.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:15.872+0000 7f47641ff700 1 -- 172.21.15.53:0/1376917517 wait complete. 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:18:15.892 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:18:15.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:15.893 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-28T11:18:15.939 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-28T11:18:15.940 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42149sec preferred_lft 42149sec 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::8530:f080:ccdf:ff7b/64 scope link noprefixroute 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether f6:19:17:2a:a2:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.941 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::487e:78ff:fe3b:3f42/64 scope link 2023-11-28T11:18:15.942 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.942 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:18:15.942 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether f6:19:17:2a:a2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:18:15.942 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::f419:17ff:fe2a:a2e4/64 scope link 2023-11-28T11:18:15.942 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:15.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:15.943 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:18:16.052 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-28T11:18:16.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.055 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-28T11:18:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-28T11:18:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:18:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:18:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:18:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:18:16.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:18:16.103 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:16.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.315 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:18:16.358 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:18:16.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.360 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-11-28T11:18:16.443 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:18:16.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.444 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:16.444 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:16.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:18:16.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:18:16.444 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:16.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 6a:ee:36:4c:ac:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:18:16.697 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:18:16.698 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.698 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::68ee:36ff:fe4c:ac06/64 scope link 2023-11-28T11:18:16.698 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.700 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:18:16.768 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 6a:ee:36:4c:ac:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::68ee:36ff:fe4c:ac06/64 scope link 2023-11-28T11:18:16.770 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:18:16.771 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:18:16.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:16.771 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:18:16.772 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:17.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.022 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-28T11:18:17.023 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-28T11:18:17.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2023-11-28T11:18:17.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:18:17.023 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-28T11:18:17.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:17.196 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:18:17.239 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:18:17.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:18:17.240 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:18:17.323 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:18:17.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:18:17.468 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:18:17.468 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:18:17.468 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:17.487 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:18:17.487 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:18:17.487 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:17.487 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:17.487 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:17.510 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:18:17.511 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:18:17.511 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2023-11-28T11:18:17.511 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:18:17.511 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:18:17.511 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:18:17.512 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:18:17.512 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:18:17.516 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:18:17.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:18:17.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.585 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:17.671 INFO:teuthology.orchestra.run.smithi157.stdout:client4645 2023-11-28T11:18:17.671 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:17.671 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:17.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.714 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:17.843 INFO:teuthology.orchestra.run.smithi157.stdout:client4645 2023-11-28T11:18:17.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.845 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:18:17.931 INFO:teuthology.orchestra.run.smithi157.stdout:client4645 2023-11-28T11:18:17.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:17.931 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/38901cf1-eb03-4cf6-8c7d-44ad237eb202.client4645/status') 2023-11-28T11:18:18.038 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4645 (3)192.168.0.2:0/355642188 2023-11-28T11:18:18.038 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2023-11-28T11:18:18.039 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2023-11-28T11:18:18.039 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2023-11-28T11:18:18.039 INFO:teuthology.orchestra.run.smithi157.stderr:68 bytes copied, 5.3331e-05 s, 1.3 MB/s 2023-11-28T11:18:18.042 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:18:18.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:18.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.270+0000 7f8a81bff700 1 Processor -- start 2023-11-28T11:18:18.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.270+0000 7f8a81bff700 1 -- start start 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a81bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a7c15da70 0x7f8a7c164500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a81bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c15e0c0 con 0x7f8a7c0c9cd0 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a81bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c15de40 con 0x7f8a7c1595a0 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a81bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c0cd030 con 0x7f8a7c15da70 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a7bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a7c15da70 0x7f8a7c164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a73fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8a7c0c9cd0 0x7f8a7c15d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48790/0 (socket says 172.21.15.53:48790) 2023-11-28T11:18:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.271+0000 7f8a73fff700 1 -- 172.21.15.53:0/3016645768 learned_addr learned my addr 172.21.15.53:0/3016645768 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7bfff700 1 -- 172.21.15.53:0/3016645768 >> v1:172.21.15.157:6789/0 conn(0x7f8a7c1595a0 legacy=0x7f8a7c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7bfff700 1 -- 172.21.15.53:0/3016645768 >> v1:172.21.15.53:6789/0 conn(0x7f8a7c0c9cd0 legacy=0x7f8a7c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7bfff700 1 -- 172.21.15.53:0/3016645768 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0cd310 con 0x7f8a7c15da70 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7affd700 1 -- 172.21.15.53:0/3016645768 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277258969 0 0) 0x7f8a7c15e0c0 con 0x7f8a7c0c9cd0 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7bfff700 1 --2- 172.21.15.53:0/3016645768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c164500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009d90 tx=0x7f8a7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd56572069a63e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7affd700 1 -- 172.21.15.53:0/3016645768 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400e070 con 0x7f8a7c15da70 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7affd700 1 -- 172.21.15.53:0/3016645768 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400bbd0 con 0x7f8a7c15da70 2023-11-28T11:18:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.272+0000 7f8a7affd700 1 -- 172.21.15.53:0/3016645768 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74012430 con 0x7f8a7c15da70 2023-11-28T11:18:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 -- 172.21.15.53:0/3016645768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 msgr2=0x7f8a7c164500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 --2- 172.21.15.53:0/3016645768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c164500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009d90 tx=0x7f8a7400b330 comp rx=0 tx=0).stop 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 -- 172.21.15.53:0/3016645768 shutdown_connections 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 --2- 172.21.15.53:0/3016645768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c164500 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 -- 172.21.15.53:0/3016645768 >> 172.21.15.53:0/3016645768 conn(0x7f8a7c0cb750 msgr2=0x7f8a7c0cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 -- 172.21.15.53:0/3016645768 shutdown_connections 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.273+0000 7f8a81bff700 1 -- 172.21.15.53:0/3016645768 wait complete. 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.275+0000 7f8a81bff700 1 Processor -- start 2023-11-28T11:18:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.275+0000 7f8a81bff700 1 -- start start 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.275+0000 7f8a81bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 0x7f8a7c146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a81bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c1595a0 0x7f8a7c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a81bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a81bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1f6670 con 0x7f8a7c15da70 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a81bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c14a460 con 0x7f8a7c1595a0 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a81bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1496a0 con 0x7f8a7c0c9cd0 2023-11-28T11:18:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54020/0 (socket says 172.21.15.53:54020) 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 -- 172.21.15.53:0/2600833538 learned_addr learned my addr 172.21.15.53:0/2600833538 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a73fff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c1595a0 0x7f8a7c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7b7fe700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 0x7f8a7c146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 -- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 msgr2=0x7f8a7c146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 0x7f8a7c146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 -- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c1595a0 msgr2=0x7f8a7c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c1595a0 0x7f8a7c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7bfff700 1 -- 172.21.15.53:0/2600833538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74009a70 con 0x7f8a7c15da70 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.276+0000 7f8a7b7fe700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 0x7f8a7c146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a7bfff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a74014040 tx=0x7f8a74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401b820 con 0x7f8a7c15da70 2023-11-28T11:18:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74020070 con 0x7f8a7c15da70 2023-11-28T11:18:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0cd8d0 con 0x7f8a7c15da70 2023-11-28T11:18:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401bbc0 con 0x7f8a7c15da70 2023-11-28T11:18:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.277+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0cd5f0 con 0x7f8a7c15da70 2023-11-28T11:18:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.278+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c070c70 con 0x7f8a7c15da70 2023-11-28T11:18:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.279+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a7401bd20 con 0x7f8a7c15da70 2023-11-28T11:18:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.280+0000 7f8a78ff9700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 0x7f8a5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.280+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a74087e90 con 0x7f8a7c15da70 2023-11-28T11:18:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.281+0000 7f8a7b7fe700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 0x7f8a5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.281+0000 7f8a7b7fe700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 0x7f8a5c06a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000f60 tx=0x7f8a6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.285+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8a74054600 con 0x7f8a7c15da70 2023-11-28T11:18:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.486+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a7c0745c0 con 0x7f8a7c15da70 2023-11-28T11:18:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.487+0000 7f8a78ff9700 1 -- 172.21.15.53:0/2600833538 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f8a74022090 con 0x7f8a7c15da70 2023-11-28T11:18:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:18.490 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 msgr2=0x7f8a5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 0x7f8a5c06a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000f60 tx=0x7f8a6c009040 comp rx=0 tx=0).stop 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 msgr2=0x7f8a7c0c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a74014040 tx=0x7f8a74004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 shutdown_connections 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a5c0682f0 0x7f8a5c06a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0c9cd0 0x7f8a7c146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a7c15da70 0x7f8a7c0c82f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 --2- 172.21.15.53:0/2600833538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c1595a0 0x7f8a7c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 >> 172.21.15.53:0/2600833538 conn(0x7f8a7c0cb750 msgr2=0x7f8a7c0cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 shutdown_connections 2023-11-28T11:18:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.490+0000 7f8a81bff700 1 -- 172.21.15.53:0/2600833538 wait complete. 2023-11-28T11:18:18.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:18.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.713+0000 7fcf78bff700 1 Processor -- start 2023-11-28T11:18:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.713+0000 7fcf78bff700 1 -- start start 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf78bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf7415c9b0 0x7fcf740b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf78bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcf740cd7c0 0x7fcf740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf78bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740d1950 con 0x7fcf74161340 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf78bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740b5800 con 0x7fcf740cd7c0 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf78bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740ce140 con 0x7fcf7415c9b0 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf71d9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcf74161340 0x7fcf740d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48804/0 (socket says 172.21.15.53:48804) 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf71d9b700 1 -- 172.21.15.53:0/4127551038 learned_addr learned my addr 172.21.15.53:0/4127551038 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.714+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/4127551038 >> v2:172.21.15.53:3300/0 conn(0x7fcf740cd7c0 0x7fcf740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf7259c700 1 --2- 172.21.15.53:0/4127551038 >> v2:172.21.15.157:3300/0 conn(0x7fcf7415c9b0 0x7fcf740b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf7159a700 1 -- 172.21.15.53:0/4127551038 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344773363 0 0) 0x7fcf740d1950 con 0x7fcf74161340 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf72d9d700 1 -- 172.21.15.53:0/4127551038 >> v2:172.21.15.157:3300/0 conn(0x7fcf7415c9b0 msgr2=0x7fcf740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/4127551038 >> v2:172.21.15.157:3300/0 conn(0x7fcf7415c9b0 0x7fcf740b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf72d9d700 1 -- 172.21.15.53:0/4127551038 >> v1:172.21.15.53:6789/0 conn(0x7fcf74161340 legacy=0x7fcf740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf72d9d700 1 -- 172.21.15.53:0/4127551038 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740d24f0 con 0x7fcf740cd7c0 2023-11-28T11:18:18.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/4127551038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf740cd7c0 0x7fcf740cdc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf74067460 tx=0x7fcf6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33b690595ece0b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.715+0000 7fcf7159a700 1 -- 172.21.15.53:0/4127551038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e070 con 0x7fcf740cd7c0 2023-11-28T11:18:18.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf7159a700 1 -- 172.21.15.53:0/4127551038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00bb20 con 0x7fcf740cd7c0 2023-11-28T11:18:18.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf7159a700 1 -- 172.21.15.53:0/4127551038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c012430 con 0x7fcf740cd7c0 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 -- 172.21.15.53:0/4127551038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf740cd7c0 msgr2=0x7fcf740cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 --2- 172.21.15.53:0/4127551038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf740cd7c0 0x7fcf740cdc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf74067460 tx=0x7fcf6c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 -- 172.21.15.53:0/4127551038 shutdown_connections 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 --2- 172.21.15.53:0/4127551038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf740cd7c0 0x7fcf740cdc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 --2- 172.21.15.53:0/4127551038 >> v2:172.21.15.157:3300/0 conn(0x7fcf7415c9b0 0x7fcf740b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 -- 172.21.15.53:0/4127551038 >> 172.21.15.53:0/4127551038 conn(0x7fcf74072600 msgr2=0x7fcf740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 -- 172.21.15.53:0/4127551038 shutdown_connections 2023-11-28T11:18:18.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.716+0000 7fcf78bff700 1 -- 172.21.15.53:0/4127551038 wait complete. 2023-11-28T11:18:18.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.718+0000 7fcf78bff700 1 Processor -- start 2023-11-28T11:18:18.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.718+0000 7fcf78bff700 1 -- start start 2023-11-28T11:18:18.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf740cd7c0 0x7fcf741f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 0x7fcf740cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74149fd0 con 0x7fcf74161340 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf741f5ec0 con 0x7fcf740cd7c0 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf78bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74159910 con 0x7fcf7415c9b0 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf72d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf72d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54030/0 (socket says 172.21.15.53:54030) 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.719+0000 7fcf72d9d700 1 -- 172.21.15.53:0/1358874593 learned_addr learned my addr 172.21.15.53:0/1358874593 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf7259c700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf740cd7c0 0x7fcf741f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf71d9b700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 0x7fcf740cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 -- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 msgr2=0x7fcf740cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 0x7fcf740cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 -- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf740cd7c0 msgr2=0x7fcf741f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf740cd7c0 0x7fcf741f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 -- 172.21.15.53:0/1358874593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009a70 con 0x7fcf74161340 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf71d9b700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 0x7fcf740cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf72d9d700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c014040 tx=0x7fcf6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c012430 con 0x7fcf74161340 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c020070 con 0x7fcf74161340 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740d1300 con 0x7fcf74161340 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004280 con 0x7fcf74161340 2023-11-28T11:18:18.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.720+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740b5520 con 0x7fcf74161340 2023-11-28T11:18:18.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.722+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf740d15e0 con 0x7fcf74161340 2023-11-28T11:18:18.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.722+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf6c02e070 con 0x7fcf74161340 2023-11-28T11:18:18.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.723+0000 7fcf62ffd700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 0x7fcf5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:18.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.723+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf6c022070 con 0x7fcf74161340 2023-11-28T11:18:18.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.724+0000 7fcf7259c700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 0x7fcf5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:18.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.725+0000 7fcf7259c700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 0x7fcf5406a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009ad0 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.728+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf6c054240 con 0x7fcf74161340 2023-11-28T11:18:18.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.930+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf7414a110 con 0x7fcf74161340 2023-11-28T11:18:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.930+0000 7fcf62ffd700 1 -- 172.21.15.53:0/1358874593 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf6c02c020 con 0x7fcf74161340 2023-11-28T11:18:18.933 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 msgr2=0x7fcf5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 0x7fcf5406a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009ad0 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 msgr2=0x7fcf740cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c014040 tx=0x7fcf6c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 shutdown_connections 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcf540682f0 0x7fcf5406a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf74161340 0x7fcf740cb900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf740cd7c0 0x7fcf741f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 --2- 172.21.15.53:0/1358874593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7415c9b0 0x7fcf740cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 >> 172.21.15.53:0/1358874593 conn(0x7fcf74072600 msgr2=0x7fcf740d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 shutdown_connections 2023-11-28T11:18:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:18.933+0000 7fcf78bff700 1 -- 172.21.15.53:0/1358874593 wait complete. 2023-11-28T11:18:18.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:19.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.157+0000 7fb316fff700 1 Processor -- start 2023-11-28T11:18:19.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.157+0000 7fb316fff700 1 -- start start 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.157+0000 7fb316fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb31015c9b0 0x7fb3100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.157+0000 7fb316fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3100cd7c0 0x7fb3100cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb316fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3100d1950 con 0x7fb310161340 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb316fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100b5800 con 0x7fb31015c9b0 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb316fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100ce140 con 0x7fb3100cd7c0 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb31015c9b0 0x7fb3100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb31015c9b0 0x7fb3100b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54040/0 (socket says 172.21.15.53:54040) 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 -- 172.21.15.53:0/106402269 learned_addr learned my addr 172.21.15.53:0/106402269 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb31559c700 1 --2- 172.21.15.53:0/106402269 >> v2:172.21.15.157:3300/0 conn(0x7fb3100cd7c0 0x7fb3100cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 -- 172.21.15.53:0/106402269 >> v2:172.21.15.157:3300/0 conn(0x7fb3100cd7c0 msgr2=0x7fb3100cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 --2- 172.21.15.53:0/106402269 >> v2:172.21.15.157:3300/0 conn(0x7fb3100cd7c0 0x7fb3100cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 -- 172.21.15.53:0/106402269 >> v1:172.21.15.157:6789/0 conn(0x7fb310161340 legacy=0x7fb3100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.158+0000 7fb314d9b700 1 -- 172.21.15.53:0/106402269 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100d24f0 con 0x7fb31015c9b0 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb314d9b700 1 --2- 172.21.15.53:0/106402269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015c9b0 0x7fb3100b4bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb31006c170 tx=0x7fb304009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a94a917c18085db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb30f7fe700 1 -- 172.21.15.53:0/106402269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400b870 con 0x7fb31015c9b0 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb30f7fe700 1 -- 172.21.15.53:0/106402269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400b9d0 con 0x7fb31015c9b0 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb30f7fe700 1 -- 172.21.15.53:0/106402269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400bd60 con 0x7fb31015c9b0 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 -- 172.21.15.53:0/106402269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015c9b0 msgr2=0x7fb3100b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 --2- 172.21.15.53:0/106402269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015c9b0 0x7fb3100b4bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb31006c170 tx=0x7fb304009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 -- 172.21.15.53:0/106402269 shutdown_connections 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 --2- 172.21.15.53:0/106402269 >> v2:172.21.15.157:3300/0 conn(0x7fb3100cd7c0 0x7fb3100cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 --2- 172.21.15.53:0/106402269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015c9b0 0x7fb3100b4bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.159+0000 7fb316fff700 1 -- 172.21.15.53:0/106402269 >> 172.21.15.53:0/106402269 conn(0x7fb310072600 msgr2=0x7fb3100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.160+0000 7fb316fff700 1 -- 172.21.15.53:0/106402269 shutdown_connections 2023-11-28T11:18:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.160+0000 7fb316fff700 1 -- 172.21.15.53:0/106402269 wait complete. 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.161+0000 7fb316fff700 1 Processor -- start 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.161+0000 7fb316fff700 1 -- start start 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb31015c9b0 0x7fb3100cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 0x7fb31014a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100cbce0 con 0x7fb3100cd7c0 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310156ef0 con 0x7fb310161340 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb316fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101548e0 con 0x7fb31015c9b0 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.162+0000 7fb314d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54056/0 (socket says 172.21.15.53:54056) 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 -- 172.21.15.53:0/246126019 learned_addr learned my addr 172.21.15.53:0/246126019 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb31559c700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 0x7fb31014a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb30ffff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb31015c9b0 0x7fb3100cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 -- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb31015c9b0 msgr2=0x7fb3100cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb31015c9b0 0x7fb3100cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 -- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 msgr2=0x7fb31014a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 0x7fb31014a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 -- 172.21.15.53:0/246126019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009550 con 0x7fb3100cd7c0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb31559c700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 0x7fb31014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb314d9b700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb304000c00 tx=0x7fb304004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304004520 con 0x7fb3100cd7c0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30401e070 con 0x7fb3100cd7c0 2023-11-28T11:18:19.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.163+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040038a0 con 0x7fb3100cd7c0 2023-11-28T11:18:19.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.164+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3100d1300 con 0x7fb3100cd7c0 2023-11-28T11:18:19.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.164+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3100b5520 con 0x7fb3100cd7c0 2023-11-28T11:18:19.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.165+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3100d15e0 con 0x7fb3100cd7c0 2023-11-28T11:18:19.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.165+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb30400f050 con 0x7fb3100cd7c0 2023-11-28T11:18:19.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.167+0000 7fb30d7fa700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 0x7fb2f406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.167+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb304020070 con 0x7fb3100cd7c0 2023-11-28T11:18:19.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.167+0000 7fb30ffff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 0x7fb2f406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.168+0000 7fb30ffff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 0x7fb2f406a7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b8e20 tx=0x7fb2fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.171+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3040533f0 con 0x7fb3100cd7c0 2023-11-28T11:18:19.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.372+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb310159a70 con 0x7fb3100cd7c0 2023-11-28T11:18:19.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.372+0000 7fb30d7fa700 1 -- 172.21.15.53:0/246126019 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb3040150a0 con 0x7fb3100cd7c0 2023-11-28T11:18:19.374 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 msgr2=0x7fb2f406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 0x7fb2f406a7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b8e20 tx=0x7fb2fc009040 comp rx=0 tx=0).stop 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 msgr2=0x7fb3100ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb304000c00 tx=0x7fb304004750 comp rx=0 tx=0).stop 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 shutdown_connections 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2f4068320 0x7fb2f406a7d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb310161340 0x7fb31014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3100cd7c0 0x7fb3100ca240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 --2- 172.21.15.53:0/246126019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb31015c9b0 0x7fb3100cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 >> 172.21.15.53:0/246126019 conn(0x7fb310072600 msgr2=0x7fb310159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:19.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 shutdown_connections 2023-11-28T11:18:19.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.375+0000 7fb316fff700 1 -- 172.21.15.53:0/246126019 wait complete. 2023-11-28T11:18:19.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:19.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.598+0000 7f745dfff700 1 Processor -- start 2023-11-28T11:18:19.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.598+0000 7f745dfff700 1 -- start start 2023-11-28T11:18:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f745dfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f745815c9b0 0x7f74580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f745dfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74580cd7c0 0x7f74580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f745dfff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580d1950 con 0x7f7458161340 2023-11-28T11:18:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f745dfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580b5800 con 0x7f745815c9b0 2023-11-28T11:18:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f745dfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ce140 con 0x7f74580cd7c0 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f745815c9b0 0x7f74580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f745815c9b0 0x7f74580b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54062/0 (socket says 172.21.15.53:54062) 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 -- 172.21.15.53:0/2753916692 learned_addr learned my addr 172.21.15.53:0/2753916692 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f7457fff700 1 --2- 172.21.15.53:0/2753916692 >> v2:172.21.15.157:3300/0 conn(0x7f74580cd7c0 0x7f74580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 -- 172.21.15.53:0/2753916692 >> v2:172.21.15.157:3300/0 conn(0x7f74580cd7c0 msgr2=0x7f74580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 --2- 172.21.15.53:0/2753916692 >> v2:172.21.15.157:3300/0 conn(0x7f74580cd7c0 0x7f74580cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 -- 172.21.15.53:0/2753916692 >> v1:172.21.15.157:6789/0 conn(0x7f7458161340 legacy=0x7f74580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 -- 172.21.15.53:0/2753916692 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580d24f0 con 0x7f745815c9b0 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f74577fe700 1 --2- 172.21.15.53:0/2753916692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f74580b4bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f745806c170 tx=0x7f7448009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd54f589d47818 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f7456ffd700 1 -- 172.21.15.53:0/2753916692 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800b870 con 0x7f745815c9b0 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f7456ffd700 1 -- 172.21.15.53:0/2753916692 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744800b9d0 con 0x7f745815c9b0 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.600+0000 7f7456ffd700 1 -- 172.21.15.53:0/2753916692 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800bd60 con 0x7f745815c9b0 2023-11-28T11:18:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 -- 172.21.15.53:0/2753916692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 msgr2=0x7f74580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 --2- 172.21.15.53:0/2753916692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f74580b4bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f745806c170 tx=0x7f7448009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 -- 172.21.15.53:0/2753916692 shutdown_connections 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 --2- 172.21.15.53:0/2753916692 >> v2:172.21.15.157:3300/0 conn(0x7f74580cd7c0 0x7f74580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 --2- 172.21.15.53:0/2753916692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f74580b4bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 -- 172.21.15.53:0/2753916692 >> 172.21.15.53:0/2753916692 conn(0x7f7458072600 msgr2=0x7f74580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:19.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 -- 172.21.15.53:0/2753916692 shutdown_connections 2023-11-28T11:18:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.601+0000 7f745dfff700 1 -- 172.21.15.53:0/2753916692 wait complete. 2023-11-28T11:18:19.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.602+0000 7f745dfff700 1 Processor -- start 2023-11-28T11:18:19.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.603+0000 7f745dfff700 1 -- start start 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.603+0000 7f745dfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74580cd7c0 0x7f7458157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f745dfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f745dfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7458161340 0x7f7458156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f745dfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745815f5a0 con 0x7f745815c9b0 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f745dfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745815ef70 con 0x7f74580cd7c0 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f745dfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745815ff70 con 0x7f7458161340 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54074/0 (socket says 172.21.15.53:54074) 2023-11-28T11:18:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 -- 172.21.15.53:0/3819225739 learned_addr learned my addr 172.21.15.53:0/3819225739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f74577fe700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74580cd7c0 0x7f7458157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f7457fff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7458161340 0x7f7458156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 -- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7458161340 msgr2=0x7f7458156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7458161340 0x7f7458156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 -- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74580cd7c0 msgr2=0x7f7458157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74580cd7c0 0x7f7458157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.604+0000 7f744ffff700 1 -- 172.21.15.53:0/3819225739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448009550 con 0x7f745815c9b0 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f744ffff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f744000c450 tx=0x7f744000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400052d0 con 0x7f745815c9b0 2023-11-28T11:18:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440005430 con 0x7f745815c9b0 2023-11-28T11:18:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580d1300 con 0x7f745815c9b0 2023-11-28T11:18:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400038c0 con 0x7f745815c9b0 2023-11-28T11:18:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.605+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580b5520 con 0x7f745815c9b0 2023-11-28T11:18:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.606+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74580d15e0 con 0x7f745815c9b0 2023-11-28T11:18:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.608+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7440019030 con 0x7f745815c9b0 2023-11-28T11:18:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.609+0000 7f7454ff9700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 0x7f743806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.609+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7440084f20 con 0x7f745815c9b0 2023-11-28T11:18:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.609+0000 7f74577fe700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 0x7f743806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.609+0000 7f74577fe700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 0x7f743806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7448014040 tx=0x7f7448008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:19.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.612+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7440051690 con 0x7f745815c9b0 2023-11-28T11:18:19.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.816+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74580745c0 con 0x7f745815c9b0 2023-11-28T11:18:19.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.816+0000 7f7454ff9700 1 -- 172.21.15.53:0/3819225739 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7440017020 con 0x7f745815c9b0 2023-11-28T11:18:19.817 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 msgr2=0x7f743806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 0x7f743806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7448014040 tx=0x7f7448008030 comp rx=0 tx=0).stop 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 msgr2=0x7f745815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f744000c450 tx=0x7f744000c760 comp rx=0 tx=0).stop 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 shutdown_connections 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f74380682f0 0x7f743806a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7458161340 0x7f7458156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74580cd7c0 0x7f7458157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 --2- 172.21.15.53:0/3819225739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f745815c9b0 0x7f745815fa30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 >> 172.21.15.53:0/3819225739 conn(0x7f7458072600 msgr2=0x7f74580b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 shutdown_connections 2023-11-28T11:18:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:19.819+0000 7f745dfff700 1 -- 172.21.15.53:0/3819225739 wait complete. 2023-11-28T11:18:19.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:20.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.043+0000 7f10f71ff700 1 Processor -- start 2023-11-28T11:18:20.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.043+0000 7f10f71ff700 1 -- start start 2023-11-28T11:18:20.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.044+0000 7f10f71ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f015c9b0 0x7f10f00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.044+0000 7f10f71ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10f0161340 0x7f10f00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.044+0000 7f10f71ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00d1950 con 0x7f10f00d2320 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.044+0000 7f10f71ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00b5800 con 0x7f10f0161340 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.044+0000 7f10f71ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00b5520 con 0x7f10f015c9b0 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10f579c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10f00d2320 0x7f10f00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48818/0 (socket says 172.21.15.53:48818) 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10f579c700 1 -- 172.21.15.53:0/1867194997 learned_addr learned my addr 172.21.15.53:0/1867194997 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10f4f9b700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f015c9b0 0x7f10f00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10effff700 1 --2- 172.21.15.53:0/1867194997 >> v2:172.21.15.157:3300/0 conn(0x7f10f0161340 0x7f10f00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10ef7fe700 1 -- 172.21.15.53:0/1867194997 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129740654 0 0) 0x7f10f00d1950 con 0x7f10f00d2320 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10ef7fe700 1 -- 172.21.15.53:0/1867194997 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e00035b0 con 0x7f10f00d2320 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10effff700 1 -- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f015c9b0 msgr2=0x7f10f00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10effff700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f015c9b0 0x7f10f00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10effff700 1 -- 172.21.15.53:0/1867194997 >> v1:172.21.15.53:6789/0 conn(0x7f10f00d2320 legacy=0x7f10f00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.045+0000 7f10effff700 1 -- 172.21.15.53:0/1867194997 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00d1300 con 0x7f10f0161340 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10effff700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f0161340 0x7f10f00d1de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009840 tx=0x7f10dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ff17b9a4e182ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10ef7fe700 1 -- 172.21.15.53:0/1867194997 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc011830 con 0x7f10f0161340 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10ef7fe700 1 -- 172.21.15.53:0/1867194997 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc011990 con 0x7f10f0161340 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10ef7fe700 1 -- 172.21.15.53:0/1867194997 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc011d20 con 0x7f10f0161340 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 -- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f0161340 msgr2=0x7f10f00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f0161340 0x7f10f00d1de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009840 tx=0x7f10dc009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 -- 172.21.15.53:0/1867194997 shutdown_connections 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f015c9b0 0x7f10f00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 --2- 172.21.15.53:0/1867194997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f0161340 0x7f10f00d1de0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 -- 172.21.15.53:0/1867194997 >> 172.21.15.53:0/1867194997 conn(0x7f10f0072600 msgr2=0x7f10f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.046+0000 7f10f71ff700 1 -- 172.21.15.53:0/1867194997 shutdown_connections 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.047+0000 7f10f71ff700 1 -- 172.21.15.53:0/1867194997 wait complete. 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.048+0000 7f10f71ff700 1 Processor -- start 2023-11-28T11:18:20.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.048+0000 7f10f71ff700 1 -- start start 2023-11-28T11:18:20.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f015c9b0 0x7f10f01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f0161340 0x7f10f01f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0151c30 con 0x7f10f00d2320 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0151950 con 0x7f10f015c9b0 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f71ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00c8240 con 0x7f10f0161340 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.049+0000 7f10f4f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54082/0 (socket says 172.21.15.53:54082) 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 -- 172.21.15.53:0/2142073197 learned_addr learned my addr 172.21.15.53:0/2142073197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10effff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f015c9b0 0x7f10f01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f579c700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f0161340 0x7f10f01f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 -- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f0161340 msgr2=0x7f10f01f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f0161340 0x7f10f01f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 -- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f015c9b0 msgr2=0x7f10f01f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f015c9b0 0x7f10f01f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 -- 172.21.15.53:0/2142073197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc0094a0 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10f4f9b700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e870 tx=0x7f10e400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.050+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4009590 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.051+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4013680 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.051+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e40139f0 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.051+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00b5130 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.051+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00d15e0 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.052+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f00b52c0 con 0x7f10f00d2320 2023-11-28T11:18:20.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.053+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f10e4019030 con 0x7f10f00d2320 2023-11-28T11:18:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.054+0000 7f10ed7fa700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 0x7f10d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.054+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10e4085a10 con 0x7f10f00d2320 2023-11-28T11:18:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.054+0000 7f10effff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 0x7f10d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.055+0000 7f10effff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 0x7f10d406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.058+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f10e4052270 con 0x7f10f00d2320 2023-11-28T11:18:20.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.260+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10f00745c0 con 0x7f10f00d2320 2023-11-28T11:18:20.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.260+0000 7f10ed7fa700 1 -- 172.21.15.53:0/2142073197 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10e4055890 con 0x7f10f00d2320 2023-11-28T11:18:20.262 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 msgr2=0x7f10d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 0x7f10d406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc008030 comp rx=0 tx=0).stop 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 msgr2=0x7f10f01507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e870 tx=0x7f10e400eb80 comp rx=0 tx=0).stop 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 shutdown_connections 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f10d40682f0 0x7f10d406a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f0161340 0x7f10f01f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10f00d2320 0x7f10f01507a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 --2- 172.21.15.53:0/2142073197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f015c9b0 0x7f10f01f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 >> 172.21.15.53:0/2142073197 conn(0x7f10f0072600 msgr2=0x7f10f015fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:20.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 shutdown_connections 2023-11-28T11:18:20.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.263+0000 7f10f71ff700 1 -- 172.21.15.53:0/2142073197 wait complete. 2023-11-28T11:18:20.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:20.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.482+0000 7fc21d3ff700 1 Processor -- start 2023-11-28T11:18:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.483+0000 7fc21d3ff700 1 -- start start 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc21d3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc21815c9b0 0x7fc2180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc21d3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2180cd7c0 0x7fc2180cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc21d3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180d1950 con 0x7fc218161340 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc21d3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180b5800 con 0x7fc21815c9b0 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc21d3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180ce140 con 0x7fc2180cd7c0 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc216ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc21815c9b0 0x7fc2180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc20ffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc218161340 0x7fc2180d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48822/0 (socket says 172.21.15.53:48822) 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc20ffff700 1 -- 172.21.15.53:0/2805265288 learned_addr learned my addr 172.21.15.53:0/2805265288 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.484+0000 7fc2177fe700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2180cd7c0 0x7fc2180cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc2167fc700 1 -- 172.21.15.53:0/2805265288 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114001334 0 0) 0x7fc2180d1950 con 0x7fc218161340 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc2167fc700 1 -- 172.21.15.53:0/2805265288 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2040035b0 con 0x7fc218161340 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc216ffd700 1 -- 172.21.15.53:0/2805265288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2180cd7c0 msgr2=0x7fc2180cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc216ffd700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2180cd7c0 0x7fc2180cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc216ffd700 1 -- 172.21.15.53:0/2805265288 >> v1:172.21.15.53:6789/0 conn(0x7fc218161340 legacy=0x7fc2180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc216ffd700 1 -- 172.21.15.53:0/2805265288 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180d24f0 con 0x7fc21815c9b0 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc216ffd700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc21815c9b0 0x7fc2180b4bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc208009840 tx=0x7fc208009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ef490cf58ec5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc2167fc700 1 -- 172.21.15.53:0/2805265288 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208011830 con 0x7fc21815c9b0 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc2167fc700 1 -- 172.21.15.53:0/2805265288 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208011990 con 0x7fc21815c9b0 2023-11-28T11:18:20.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.485+0000 7fc2167fc700 1 -- 172.21.15.53:0/2805265288 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208011d00 con 0x7fc21815c9b0 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 -- 172.21.15.53:0/2805265288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc21815c9b0 msgr2=0x7fc2180b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc21815c9b0 0x7fc2180b4bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc208009840 tx=0x7fc208009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 -- 172.21.15.53:0/2805265288 shutdown_connections 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2180cd7c0 0x7fc2180cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/2805265288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc21815c9b0 0x7fc2180b4bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 -- 172.21.15.53:0/2805265288 >> 172.21.15.53:0/2805265288 conn(0x7fc218072600 msgr2=0x7fc2180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 -- 172.21.15.53:0/2805265288 shutdown_connections 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.486+0000 7fc21d3ff700 1 -- 172.21.15.53:0/2805265288 wait complete. 2023-11-28T11:18:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.488+0000 7fc21d3ff700 1 Processor -- start 2023-11-28T11:18:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.488+0000 7fc21d3ff700 1 -- start start 2023-11-28T11:18:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2180cd7c0 0x7fc2180cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc21815c9b0 0x7fc2180cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180cc220 con 0x7fc218161340 2023-11-28T11:18:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180cb290 con 0x7fc2180cd7c0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc21d3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2181e6620 con 0x7fc21815c9b0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc2177fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc2177fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54104/0 (socket says 172.21.15.53:54104) 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc2177fe700 1 -- 172.21.15.53:0/791006049 learned_addr learned my addr 172.21.15.53:0/791006049 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc216ffd700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2180cd7c0 0x7fc2180cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.489+0000 7fc20ffff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc21815c9b0 0x7fc2180cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 -- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc21815c9b0 msgr2=0x7fc2180cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc21815c9b0 0x7fc2180cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 -- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2180cd7c0 msgr2=0x7fc2180cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2180cd7c0 0x7fc2180cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 -- 172.21.15.53:0/791006049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080094a0 con 0x7fc218161340 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc2177fe700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc21000efc0 tx=0x7fc21000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000cad0 con 0x7fc218161340 2023-11-28T11:18:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000f040 con 0x7fc218161340 2023-11-28T11:18:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180d1300 con 0x7fc218161340 2023-11-28T11:18:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ce70 con 0x7fc218161340 2023-11-28T11:18:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.490+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180b5520 con 0x7fc218161340 2023-11-28T11:18:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.491+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2180d15e0 con 0x7fc218161340 2023-11-28T11:18:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.492+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc210014480 con 0x7fc218161340 2023-11-28T11:18:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.493+0000 7fc20f7fe700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 0x7fc1f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.493+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2100867f0 con 0x7fc218161340 2023-11-28T11:18:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.494+0000 7fc216ffd700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 0x7fc1f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.494+0000 7fc216ffd700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 0x7fc1f806a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c00 tx=0x7fc20801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.497+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc210052ff0 con 0x7fc218161340 2023-11-28T11:18:20.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.702+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2180b9180 con 0x7fc218161340 2023-11-28T11:18:20.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.703+0000 7fc20f7fe700 1 -- 172.21.15.53:0/791006049 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fc210056610 con 0x7fc218161340 2023-11-28T11:18:20.705 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:20.705 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 msgr2=0x7fc1f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 0x7fc1f806a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c00 tx=0x7fc20801c040 comp rx=0 tx=0).stop 2023-11-28T11:18:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 msgr2=0x7fc21815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc21000efc0 tx=0x7fc21000c0c0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 shutdown_connections 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc1f80682f0 0x7fc1f806a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc218161340 0x7fc21815e630 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2180cd7c0 0x7fc2180cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 --2- 172.21.15.53:0/791006049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc21815c9b0 0x7fc2180cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 >> 172.21.15.53:0/791006049 conn(0x7fc218072600 msgr2=0x7fc21815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 shutdown_connections 2023-11-28T11:18:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.706+0000 7fc21d3ff700 1 -- 172.21.15.53:0/791006049 wait complete. 2023-11-28T11:18:20.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:20.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:20.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.930+0000 7f064f1ff700 1 Processor -- start 2023-11-28T11:18:20.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.930+0000 7f064f1ff700 1 -- start start 2023-11-28T11:18:20.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0648161340 0x7f06480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 0x7f06480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064f1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480d1950 con 0x7f064815c9b0 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06480b5800 con 0x7f0648161340 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06480b5300 con 0x7f06480d2320 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f0647fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0648161340 0x7f06480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.931+0000 7f064cf9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f064815c9b0 0x7f06480b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45290/0 (socket says 172.21.15.53:45290) 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0648161340 0x7f06480d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35734/0 (socket says 172.21.15.53:35734) 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f064cf9b700 1 -- 172.21.15.53:0/1420099121 learned_addr learned my addr 172.21.15.53:0/1420099121 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f064d79c700 1 --2- 172.21.15.53:0/1420099121 >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 0x7f06480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 -- 172.21.15.53:0/1420099121 >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 msgr2=0x7f06480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:20.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 --2- 172.21.15.53:0/1420099121 >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 0x7f06480cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 -- 172.21.15.53:0/1420099121 >> v1:172.21.15.53:6789/0 conn(0x7f064815c9b0 legacy=0x7f06480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 -- 172.21.15.53:0/1420099121 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480b5520 con 0x7f0648161340 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f064d79c700 1 --2- 172.21.15.53:0/1420099121 >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 0x7f06480cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f06477fe700 1 -- 172.21.15.53:0/1420099121 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857539909 0 0) 0x7f06480d1950 con 0x7f064815c9b0 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f0647fff700 1 --2- 172.21.15.53:0/1420099121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0648161340 0x7f06480d1de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f064806c170 tx=0x7f0634009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6062fef485167b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.932+0000 7f06477fe700 1 -- 172.21.15.53:0/1420099121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011830 con 0x7f0648161340 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f06477fe700 1 -- 172.21.15.53:0/1420099121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0634011990 con 0x7f0648161340 2023-11-28T11:18:20.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f06477fe700 1 -- 172.21.15.53:0/1420099121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011d20 con 0x7f0648161340 2023-11-28T11:18:20.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 -- 172.21.15.53:0/1420099121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0648161340 msgr2=0x7f06480d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 --2- 172.21.15.53:0/1420099121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0648161340 0x7f06480d1de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f064806c170 tx=0x7f0634009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:20.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 -- 172.21.15.53:0/1420099121 shutdown_connections 2023-11-28T11:18:20.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 --2- 172.21.15.53:0/1420099121 >> v2:172.21.15.157:3300/0 conn(0x7f06480d2320 0x7f06480cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 --2- 172.21.15.53:0/1420099121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0648161340 0x7f06480d1de0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 -- 172.21.15.53:0/1420099121 >> 172.21.15.53:0/1420099121 conn(0x7f0648072600 msgr2=0x7f06480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 -- 172.21.15.53:0/1420099121 shutdown_connections 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.933+0000 7f064f1ff700 1 -- 172.21.15.53:0/1420099121 wait complete. 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.935+0000 7f064f1ff700 1 Processor -- start 2023-11-28T11:18:20.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.935+0000 7f064f1ff700 1 -- start start 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06480d2320 0x7f06480cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0648161340 0x7f06480ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c90b0 con 0x7f064815c9b0 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648157b70 con 0x7f06480d2320 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f064f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06480cae00 con 0x7f0648161340 2023-11-28T11:18:20.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.936+0000 7f0647fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35740/0 (socket says 172.21.15.53:35740) 2023-11-28T11:18:20.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 -- 172.21.15.53:0/250145683 learned_addr learned my addr 172.21.15.53:0/250145683 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:20.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f064cf9b700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06480d2320 0x7f06480cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f064d79c700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0648161340 0x7f06480ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 -- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0648161340 msgr2=0x7f06480ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0648161340 0x7f06480ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 -- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06480d2320 msgr2=0x7f06480cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06480d2320 0x7f06480cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 -- 172.21.15.53:0/250145683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340094a0 con 0x7f064815c9b0 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f0647fff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f0634004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340036a0 con 0x7f064815c9b0 2023-11-28T11:18:20.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063401f070 con 0x7f064815c9b0 2023-11-28T11:18:20.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.937+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06480d15e0 con 0x7f064815c9b0 2023-11-28T11:18:20.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.938+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634003a20 con 0x7f064815c9b0 2023-11-28T11:18:20.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.938+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06480d1300 con 0x7f064815c9b0 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.939+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064815cd80 con 0x7f064815c9b0 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.940+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f063400f050 con 0x7f064815c9b0 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.941+0000 7f06457fa700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 0x7f062c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.941+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0634021070 con 0x7f064815c9b0 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.941+0000 7f064cf9b700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 0x7f062c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:20.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.942+0000 7f064cf9b700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 0x7f062c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f60 tx=0x7f063c007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:20.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:20.945+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0634053e10 con 0x7f064815c9b0 2023-11-28T11:18:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.146+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06480d2710 con 0x7f064815c9b0 2023-11-28T11:18:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.146+0000 7f06457fa700 1 -- 172.21.15.53:0/250145683 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0634057430 con 0x7f064815c9b0 2023-11-28T11:18:21.147 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 msgr2=0x7f062c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 0x7f062c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f60 tx=0x7f063c007e70 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 msgr2=0x7f0648157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f0634004750 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 shutdown_connections 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f062c0682f0 0x7f062c06a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0648161340 0x7f06480ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06480d2320 0x7f06480cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 --2- 172.21.15.53:0/250145683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064815c9b0 0x7f0648157630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.148+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 >> 172.21.15.53:0/250145683 conn(0x7f0648072600 msgr2=0x7f064806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.149+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 shutdown_connections 2023-11-28T11:18:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.149+0000 7f064f1ff700 1 -- 172.21.15.53:0/250145683 wait complete. 2023-11-28T11:18:21.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:21.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.374+0000 7fd2c9dff700 1 Processor -- start 2023-11-28T11:18:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.374+0000 7fd2c9dff700 1 -- start start 2023-11-28T11:18:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c9dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c415c9b0 0x7fd2c40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c9dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c9dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40ce160 con 0x7fd2c4161340 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c9dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c40b5800 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c9dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40d1950 con 0x7fd2c415c9b0 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35754/0 (socket says 172.21.15.53:35754) 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.375+0000 7fd2c3fff700 1 -- 172.21.15.53:0/3477028400 learned_addr learned my addr 172.21.15.53:0/3477028400 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c37fe700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c415c9b0 0x7fd2c40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c3fff700 1 -- 172.21.15.53:0/3477028400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c415c9b0 msgr2=0x7fd2c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c3fff700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c415c9b0 0x7fd2c40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c3fff700 1 -- 172.21.15.53:0/3477028400 >> v1:172.21.15.157:6789/0 conn(0x7fd2c4161340 legacy=0x7fd2c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c3fff700 1 -- 172.21.15.53:0/3477028400 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c40d24f0 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c3fff700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b48fe661d4e305 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c27fc700 1 -- 172.21.15.53:0/3477028400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00e070 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c27fc700 1 -- 172.21.15.53:0/3477028400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc00b9c0 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.376+0000 7fd2c27fc700 1 -- 172.21.15.53:0/3477028400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00bd70 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 -- 172.21.15.53:0/3477028400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c40cd7c0 msgr2=0x7fd2c40cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 -- 172.21.15.53:0/3477028400 shutdown_connections 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c40cdc20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/3477028400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c415c9b0 0x7fd2c40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 -- 172.21.15.53:0/3477028400 >> 172.21.15.53:0/3477028400 conn(0x7fd2c4072600 msgr2=0x7fd2c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 -- 172.21.15.53:0/3477028400 shutdown_connections 2023-11-28T11:18:21.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.377+0000 7fd2c9dff700 1 -- 172.21.15.53:0/3477028400 wait complete. 2023-11-28T11:18:21.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.379+0000 7fd2c9dff700 1 Processor -- start 2023-11-28T11:18:21.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.379+0000 7fd2c9dff700 1 -- start start 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c41f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 0x7fd2c40c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c415d8c0 con 0x7fd2c415c9b0 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40cabf0 con 0x7fd2c40cd7c0 2023-11-28T11:18:21.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c9dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40cd3f0 con 0x7fd2c4161340 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35766/0 (socket says 172.21.15.53:35766) 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 -- 172.21.15.53:0/735049784 learned_addr learned my addr 172.21.15.53:0/735049784 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c37fe700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c41f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c3fff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 0x7fd2c40c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 -- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 msgr2=0x7fd2c40c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 0x7fd2c40c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 -- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 msgr2=0x7fd2c41f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c41f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c2ffd700 1 -- 172.21.15.53:0/735049784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc009b30 con 0x7fd2c415c9b0 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c37fe700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c41f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:21.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.380+0000 7fd2c3fff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 0x7fd2c40c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2c2ffd700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c450 tx=0x7fd2b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80052d0 con 0x7fd2c415c9b0 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8005430 con 0x7fd2c415c9b0 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c40d1300 con 0x7fd2c415c9b0 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80038c0 con 0x7fd2c415c9b0 2023-11-28T11:18:21.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.381+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c40b5520 con 0x7fd2c415c9b0 2023-11-28T11:18:21.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.382+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c40d15e0 con 0x7fd2c415c9b0 2023-11-28T11:18:21.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.383+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2b8019030 con 0x7fd2c415c9b0 2023-11-28T11:18:21.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.384+0000 7fd2abfff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 0x7fd2b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.384+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2b8084f20 con 0x7fd2c415c9b0 2023-11-28T11:18:21.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.384+0000 7fd2c37fe700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 0x7fd2b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.385+0000 7fd2c37fe700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 0x7fd2b006a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400ba70 tx=0x7fd2b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.388+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2b8051690 con 0x7fd2c415c9b0 2023-11-28T11:18:21.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.590+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2c40cce90 con 0x7fd2c415c9b0 2023-11-28T11:18:21.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.590+0000 7fd2abfff700 1 -- 172.21.15.53:0/735049784 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2b8054cb0 con 0x7fd2c415c9b0 2023-11-28T11:18:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 msgr2=0x7fd2b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 0x7fd2b006a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400ba70 tx=0x7fd2b4008040 comp rx=0 tx=0).stop 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 msgr2=0x7fd2c40c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c450 tx=0x7fd2b800c810 comp rx=0 tx=0).stop 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 shutdown_connections 2023-11-28T11:18:21.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2b00682f0 0x7fd2b006a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c4161340 0x7fd2c40c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c40cd7c0 0x7fd2c41f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 --2- 172.21.15.53:0/735049784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2c415c9b0 0x7fd2c40c9120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.592+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 >> 172.21.15.53:0/735049784 conn(0x7fd2c4072600 msgr2=0x7fd2c4154530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:21.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.593+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 shutdown_connections 2023-11-28T11:18:21.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.593+0000 7fd2c9dff700 1 -- 172.21.15.53:0/735049784 wait complete. 2023-11-28T11:18:21.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:21.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.816+0000 7f21715ff700 1 Processor -- start 2023-11-28T11:18:21.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.816+0000 7f21715ff700 1 -- start start 2023-11-28T11:18:21.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.817+0000 7f21715ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f216c161340 0x7f216c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.817+0000 7f21715ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.817+0000 7f21715ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0d1950 con 0x7f216c15c9b0 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.817+0000 7f21715ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0b5800 con 0x7f216c161340 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.817+0000 7f21715ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b5300 con 0x7f216c0d2320 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f216c161340 0x7f216c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f216c161340 0x7f216c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35780/0 (socket says 172.21.15.53:35780) 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 -- 172.21.15.53:0/1428768602 learned_addr learned my addr 172.21.15.53:0/1428768602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216b7fe700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 -- 172.21.15.53:0/1428768602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 msgr2=0x7f216c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 -- 172.21.15.53:0/1428768602 >> v1:172.21.15.157:6789/0 conn(0x7f216c15c9b0 legacy=0x7f216c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 -- 172.21.15.53:0/1428768602 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0b5520 con 0x7f216c161340 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.818+0000 7f216a7fc700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c161340 0x7f216c0d1de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f21600098f0 tx=0x7f2160009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5e749db2fcaa94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f2169ffb700 1 -- 172.21.15.53:0/1428768602 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011830 con 0x7f216c161340 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f2169ffb700 1 -- 172.21.15.53:0/1428768602 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011990 con 0x7f216c161340 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f2169ffb700 1 -- 172.21.15.53:0/1428768602 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011d20 con 0x7f216c161340 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 -- 172.21.15.53:0/1428768602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c161340 msgr2=0x7f216c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c161340 0x7f216c0d1de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f21600098f0 tx=0x7f2160009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 -- 172.21.15.53:0/1428768602 shutdown_connections 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 --2- 172.21.15.53:0/1428768602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c161340 0x7f216c0d1de0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 -- 172.21.15.53:0/1428768602 >> 172.21.15.53:0/1428768602 conn(0x7f216c072600 msgr2=0x7f216c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 -- 172.21.15.53:0/1428768602 shutdown_connections 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.819+0000 7f21715ff700 1 -- 172.21.15.53:0/1428768602 wait complete. 2023-11-28T11:18:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.821+0000 7f21715ff700 1 Processor -- start 2023-11-28T11:18:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.821+0000 7f21715ff700 1 -- start start 2023-11-28T11:18:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 0x7f216c1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0c6ca0 con 0x7f216c15c9b0 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1f6e60 con 0x7f216c161340 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.822+0000 7f21715ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1f7140 con 0x7f216c0d2320 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35788/0 (socket says 172.21.15.53:35788) 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 -- 172.21.15.53:0/2219398608 learned_addr learned my addr 172.21.15.53:0/2219398608 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216affd700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216b7fe700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 0x7f216c1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 -- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 msgr2=0x7f216c150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 -- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 msgr2=0x7f216c1e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 0x7f216c1e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 -- 172.21.15.53:0/2219398608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160009550 con 0x7f216c15c9b0 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216affd700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216b7fe700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 0x7f216c1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.823+0000 7f216a7fc700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2160009c40 tx=0x7f2160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.825+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600036a0 con 0x7f216c15c9b0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.825+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216001f070 con 0x7f216c15c9b0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.825+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160003a40 con 0x7f216c15c9b0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.825+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c0d15e0 con 0x7f216c15c9b0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.826+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c0d1300 con 0x7f216c15c9b0 2023-11-28T11:18:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.827+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c15cd80 con 0x7f216c15c9b0 2023-11-28T11:18:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.828+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f216000f050 con 0x7f216c15c9b0 2023-11-28T11:18:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.829+0000 7f21537fe700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 0x7f215806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.829+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2160021070 con 0x7f216c15c9b0 2023-11-28T11:18:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.829+0000 7f216affd700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 0x7f215806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.830+0000 7f216affd700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 0x7f215806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f216c1515e0 tx=0x7f215c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:21.833+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2160053eb0 con 0x7f216c15c9b0 2023-11-28T11:18:22.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.037+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f216c15d100 con 0x7f216c15c9b0 2023-11-28T11:18:22.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.037+0000 7f21537fe700 1 -- 172.21.15.53:0/2219398608 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2160015020 con 0x7f216c15c9b0 2023-11-28T11:18:22.039 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:22.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 msgr2=0x7f215806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 0x7f215806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f216c1515e0 tx=0x7f215c009040 comp rx=0 tx=0).stop 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 msgr2=0x7f216c1f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2160009c40 tx=0x7f2160004750 comp rx=0 tx=0).stop 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 shutdown_connections 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f21580682f0 0x7f215806a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f216c0d2320 0x7f216c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c161340 0x7f216c1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 --2- 172.21.15.53:0/2219398608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f216c15c9b0 0x7f216c1f63c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.039+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 >> 172.21.15.53:0/2219398608 conn(0x7f216c072600 msgr2=0x7f216c0b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.040+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 shutdown_connections 2023-11-28T11:18:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.040+0000 7f21715ff700 1 -- 172.21.15.53:0/2219398608 wait complete. 2023-11-28T11:18:22.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:22.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.264+0000 7fb05b9ff700 1 Processor -- start 2023-11-28T11:18:22.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.265+0000 7fb05b9ff700 1 -- start start 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05b9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 0x7fb0540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05b9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb054161340 0x7fb0540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05b9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540d1950 con 0x7fb0540d2320 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05b9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540b5800 con 0x7fb054161340 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05b9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540b5520 con 0x7fb05415c9b0 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb058f9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb054161340 0x7fb0540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb058f9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb054161340 0x7fb0540d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35798/0 (socket says 172.21.15.53:35798) 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb058f9a700 1 -- 172.21.15.53:0/1620379425 learned_addr learned my addr 172.21.15.53:0/1620379425 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb059f9c700 1 --1- 172.21.15.53:0/1620379425 >> v1:172.21.15.53:6789/0 conn(0x7fb0540d2320 0x7fb0540cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45294/0 (socket says 172.21.15.53:45294) 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.266+0000 7fb05979b700 1 --2- 172.21.15.53:0/1620379425 >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 0x7fb0540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb058f9a700 1 -- 172.21.15.53:0/1620379425 >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 msgr2=0x7fb0540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb058f9a700 1 --2- 172.21.15.53:0/1620379425 >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 0x7fb0540b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb058f9a700 1 -- 172.21.15.53:0/1620379425 >> v1:172.21.15.53:6789/0 conn(0x7fb0540d2320 legacy=0x7fb0540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb058f9a700 1 -- 172.21.15.53:0/1620379425 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540d1300 con 0x7fb054161340 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb05979b700 1 --2- 172.21.15.53:0/1620379425 >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 0x7fb0540b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb04bfff700 1 -- 172.21.15.53:0/1620379425 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448728644 0 0) 0x7fb0540d1950 con 0x7fb0540d2320 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb058f9a700 1 --2- 172.21.15.53:0/1620379425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb054161340 0x7fb0540d1de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb054067460 tx=0x7fb044009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667b492eb61b3b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb04bfff700 1 -- 172.21.15.53:0/1620379425 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400b870 con 0x7fb054161340 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb04bfff700 1 -- 172.21.15.53:0/1620379425 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400b9d0 con 0x7fb054161340 2023-11-28T11:18:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.267+0000 7fb04bfff700 1 -- 172.21.15.53:0/1620379425 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400bd60 con 0x7fb054161340 2023-11-28T11:18:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 -- 172.21.15.53:0/1620379425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb054161340 msgr2=0x7fb0540d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/1620379425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb054161340 0x7fb0540d1de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb054067460 tx=0x7fb044009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 -- 172.21.15.53:0/1620379425 shutdown_connections 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/1620379425 >> v2:172.21.15.157:3300/0 conn(0x7fb05415c9b0 0x7fb0540b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/1620379425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb054161340 0x7fb0540d1de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 -- 172.21.15.53:0/1620379425 >> 172.21.15.53:0/1620379425 conn(0x7fb054072600 msgr2=0x7fb0540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 -- 172.21.15.53:0/1620379425 shutdown_connections 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.268+0000 7fb05b9ff700 1 -- 172.21.15.53:0/1620379425 wait complete. 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.269+0000 7fb05b9ff700 1 Processor -- start 2023-11-28T11:18:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.270+0000 7fb05b9ff700 1 -- start start 2023-11-28T11:18:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.270+0000 7fb05b9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.270+0000 7fb05b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 0x7fb0541e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05b9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 0x7fb0540b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05b9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054143a50 con 0x7fb0540d2320 2023-11-28T11:18:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05b9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541554a0 con 0x7fb054161340 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb054155740 con 0x7fb05415c9b0 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35812/0 (socket says 172.21.15.53:35812) 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 -- 172.21.15.53:0/2053702274 learned_addr learned my addr 172.21.15.53:0/2053702274 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb059f9c700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 0x7fb0540b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb058f9a700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 0x7fb0541e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 -- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 msgr2=0x7fb0541e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 0x7fb0541e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 -- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 msgr2=0x7fb0540b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 0x7fb0540b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb05979b700 1 -- 172.21.15.53:0/2053702274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009550 con 0x7fb0540d2320 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb059f9c700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 0x7fb0540b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.271+0000 7fb058f9a700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 0x7fb0541e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb05979b700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e810 tx=0x7fb05000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050013610 con 0x7fb0540d2320 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb050013770 con 0x7fb0540d2320 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050013b20 con 0x7fb0540d2320 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540b5130 con 0x7fb0540d2320 2023-11-28T11:18:22.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.272+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540d15e0 con 0x7fb0540d2320 2023-11-28T11:18:22.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.273+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0540b52c0 con 0x7fb0540d2320 2023-11-28T11:18:22.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.274+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb050023070 con 0x7fb0540d2320 2023-11-28T11:18:22.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.275+0000 7fb049ffb700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 0x7fb03806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.275+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0500859b0 con 0x7fb0540d2320 2023-11-28T11:18:22.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.275+0000 7fb058f9a700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 0x7fb03806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.276+0000 7fb058f9a700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 0x7fb03806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb054143d20 tx=0x7fb044008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.279+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb050052120 con 0x7fb0540d2320 2023-11-28T11:18:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.480+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0540b9130 con 0x7fb0540d2320 2023-11-28T11:18:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.480+0000 7fb049ffb700 1 -- 172.21.15.53:0/2053702274 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb050055740 con 0x7fb0540d2320 2023-11-28T11:18:22.481 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.482+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 msgr2=0x7fb03806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.482+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 0x7fb03806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb054143d20 tx=0x7fb044008030 comp rx=0 tx=0).stop 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 msgr2=0x7fb054143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e810 tx=0x7fb05000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 shutdown_connections 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb038068320 0x7fb03806a7d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb054161340 0x7fb0540b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540d2320 0x7fb054143510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 --2- 172.21.15.53:0/2053702274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb05415c9b0 0x7fb0541e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 >> 172.21.15.53:0/2053702274 conn(0x7fb054072600 msgr2=0x7fb0540b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 shutdown_connections 2023-11-28T11:18:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.483+0000 7fb05b9ff700 1 -- 172.21.15.53:0/2053702274 wait complete. 2023-11-28T11:18:22.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:22.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.703+0000 7f59ca9ff700 1 Processor -- start 2023-11-28T11:18:22.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.704+0000 7f59ca9ff700 1 -- start start 2023-11-28T11:18:22.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59ca9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59c40cd7c0 0x7f59c40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59ca9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c40d1950 con 0x7f59c4161340 2023-11-28T11:18:22.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59ca9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c40ce140 con 0x7f59c415c9b0 2023-11-28T11:18:22.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59ca9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c40b5800 con 0x7f59c40cd7c0 2023-11-28T11:18:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59bb7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f59c4161340 0x7f59c40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45304/0 (socket says 172.21.15.53:45304) 2023-11-28T11:18:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59bb7fe700 1 -- 172.21.15.53:0/60190387 learned_addr learned my addr 172.21.15.53:0/60190387 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59c8f9c700 1 --2- 172.21.15.53:0/60190387 >> v2:172.21.15.157:3300/0 conn(0x7f59c40cd7c0 0x7f59c40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.705+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246236016 0 0) 0x7f59c40d1950 con 0x7f59c4161340 2023-11-28T11:18:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b00035b0 con 0x7f59c4161340 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659400001 0 0) 0x7f59b00035b0 con 0x7f59c4161340 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 >> v2:172.21.15.157:3300/0 conn(0x7f59c40cd7c0 msgr2=0x7f59c40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 --2- 172.21.15.53:0/60190387 >> v2:172.21.15.157:3300/0 conn(0x7f59c40cd7c0 0x7f59c40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 >> v1:172.21.15.157:6789/0 conn(0x7f59c415c9b0 legacy=0x7f59c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c40d24f0 con 0x7f59c4161340 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f59ac001670 con 0x7f59c4161340 2023-11-28T11:18:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 >> v1:172.21.15.53:6789/0 conn(0x7f59c4161340 legacy=0x7f59c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b0004850 0x7f59b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b0005200 0x7f59b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40b5800 con 0x7f59b0005200 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b000c110 con 0x7f59b0003a00 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b000c2d0 con 0x7f59b0004850 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c3fff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59c8f9c700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b0005200 0x7f59b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.706+0000 7f59bb7fe700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b0004850 0x7f59b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 -- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b0004850 msgr2=0x7f59b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59b0004850 0x7f59b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 -- 172.21.15.53:0/60190387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b0005200 msgr2=0x7f59b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b0005200 0x7f59b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 -- 172.21.15.53:0/60190387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00035b0 con 0x7f59b0003a00 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c8f9c700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b0005200 0x7f59b000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c8f9c700 1 -- 172.21.15.53:0/60190387 reap_dead start 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c3fff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59b0003e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f59b40017c0 tx=0x7f59b0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b6ca0648381d9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40052d0 con 0x7f59b0003a00 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4005430 con 0x7f59b0003a00 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.707+0000 7f59c37fe700 1 -- 172.21.15.53:0/60190387 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40038c0 con 0x7f59b0003a00 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 -- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 msgr2=0x7f59b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59b0003e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f59b40017c0 tx=0x7f59b0005880 comp rx=0 tx=0).stop 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 -- 172.21.15.53:0/60190387 shutdown_connections 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/60190387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59b0003e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 -- 172.21.15.53:0/60190387 >> 172.21.15.53:0/60190387 conn(0x7f59c4072600 msgr2=0x7f59c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:22.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 -- 172.21.15.53:0/60190387 shutdown_connections 2023-11-28T11:18:22.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.708+0000 7f59ca9ff700 1 -- 172.21.15.53:0/60190387 wait complete. 2023-11-28T11:18:22.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.710+0000 7f59ca9ff700 1 Processor -- start 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.710+0000 7f59ca9ff700 1 -- start start 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59c40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c4153d30 0x7f59c415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40cbac0 con 0x7f59c40cadc0 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c41f59a0 con 0x7f59b0003a00 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59ca9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4157e50 con 0x7f59c4153d30 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35822/0 (socket says 172.21.15.53:35822) 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59bb7fe700 1 -- 172.21.15.53:0/4260934265 learned_addr learned my addr 172.21.15.53:0/4260934265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59c8f9c700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c4153d30 0x7f59c415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.711+0000 7f59c3fff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59c40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 -- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c4153d30 msgr2=0x7f59c415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c4153d30 0x7f59c415e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 -- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 msgr2=0x7f59c40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59c40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 -- 172.21.15.53:0/4260934265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c40d1300 con 0x7f59c40cadc0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59c3fff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59c40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59bb7fe700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f59ac007290 tx=0x7f59ac0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00f640 con 0x7f59c40cadc0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac00f7c0 con 0x7f59c40cadc0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00fb70 con 0x7f59c40cadc0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c40b5520 con 0x7f59c40cadc0 2023-11-28T11:18:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.712+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c40d15e0 con 0x7f59c40cadc0 2023-11-28T11:18:22.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.713+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c40745c0 con 0x7f59c40cadc0 2023-11-28T11:18:22.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.714+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f59ac00fcd0 con 0x7f59c40cadc0 2023-11-28T11:18:22.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.715+0000 7f59c17fa700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 0x7f59a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:22.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.715+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59ac085de0 con 0x7f59c40cadc0 2023-11-28T11:18:22.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.716+0000 7f59c3fff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 0x7f59a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:22.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.716+0000 7f59c3fff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 0x7f59a406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59b4015040 tx=0x7f59b4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:22.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.719+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f59ac052550 con 0x7f59c40cadc0 2023-11-28T11:18:22.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.924+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c40d08a0 con 0x7f59c40cadc0 2023-11-28T11:18:22.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.925+0000 7f59c17fa700 1 -- 172.21.15.53:0/4260934265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f59ac05b030 con 0x7f59c40cadc0 2023-11-28T11:18:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:22.927 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 msgr2=0x7f59a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 0x7f59a406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59b4015040 tx=0x7f59b4009030 comp rx=0 tx=0).stop 2023-11-28T11:18:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 msgr2=0x7f59c41f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f59ac007290 tx=0x7f59ac0075a0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 shutdown_connections 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f59a40682f0 0x7f59a406a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c4153d30 0x7f59c415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b0003a00 0x7f59c40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 --2- 172.21.15.53:0/4260934265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59c40cadc0 0x7f59c41f63b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 >> 172.21.15.53:0/4260934265 conn(0x7f59c4072600 msgr2=0x7f59c4160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 shutdown_connections 2023-11-28T11:18:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:22.928+0000 7f59ca9ff700 1 -- 172.21.15.53:0/4260934265 wait complete. 2023-11-28T11:18:22.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:22.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.154+0000 7fe942bff700 1 Processor -- start 2023-11-28T11:18:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.154+0000 7fe942bff700 1 -- start start 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.155+0000 7fe942bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe93c0cd7c0 0x7fe93c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe942bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c0d1950 con 0x7fe93c161340 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe942bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c0ce140 con 0x7fe93c15c9b0 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe942bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c0b5800 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe93c161340 0x7fe93c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45316/0 (socket says 172.21.15.53:45316) 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93bfff700 1 -- 172.21.15.53:0/3613971684 learned_addr learned my addr 172.21.15.53:0/3613971684 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe94119c700 1 --2- 172.21.15.53:0/3613971684 >> v2:172.21.15.157:3300/0 conn(0x7fe93c0cd7c0 0x7fe93c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800286099 0 0) 0x7fe93c0d1950 con 0x7fe93c161340 2023-11-28T11:18:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe92c0035b0 con 0x7fe93c161340 2023-11-28T11:18:23.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274097616 0 0) 0x7fe93c0ce140 con 0x7fe93c15c9b0 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.156+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe93c0d1950 con 0x7fe93c15c9b0 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe94119c700 1 -- 172.21.15.53:0/3613971684 >> v1:172.21.15.157:6789/0 conn(0x7fe93c15c9b0 legacy=0x7fe93c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442126500 0 0) 0x7fe92c0035b0 con 0x7fe93c161340 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe94119c700 1 -- 172.21.15.53:0/3613971684 >> v1:172.21.15.53:6789/0 conn(0x7fe93c161340 legacy=0x7fe93c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe94119c700 1 -- 172.21.15.53:0/3613971684 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0d24f0 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe94119c700 1 --2- 172.21.15.53:0/3613971684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0cdc00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb34e99d30d9ed3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400e070 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe934003a20 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.157+0000 7fe93b7fe700 1 -- 172.21.15.53:0/3613971684 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928001670 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 -- 172.21.15.53:0/3613971684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 msgr2=0x7fe93c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 --2- 172.21.15.53:0/3613971684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0cdc00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93400b330 comp rx=0 tx=0).stop 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 -- 172.21.15.53:0/3613971684 shutdown_connections 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 --2- 172.21.15.53:0/3613971684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0cdc00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 -- 172.21.15.53:0/3613971684 >> 172.21.15.53:0/3613971684 conn(0x7fe93c072600 msgr2=0x7fe93c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 -- 172.21.15.53:0/3613971684 shutdown_connections 2023-11-28T11:18:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.158+0000 7fe942bff700 1 -- 172.21.15.53:0/3613971684 wait complete. 2023-11-28T11:18:23.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.160+0000 7fe942bff700 1 Processor -- start 2023-11-28T11:18:23.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.160+0000 7fe942bff700 1 -- start start 2023-11-28T11:18:23.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe93c15c9b0 0x7fe93c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0cb320 con 0x7fe93c161340 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0c7bd0 con 0x7fe93c0cd7c0 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe942bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0c6ca0 con 0x7fe93c15c9b0 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35838/0 (socket says 172.21.15.53:35838) 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 -- 172.21.15.53:0/1342784902 learned_addr learned my addr 172.21.15.53:0/1342784902 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94099b700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe93bfff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe93c15c9b0 0x7fe93c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 -- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe93c15c9b0 msgr2=0x7fe93c14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe93c15c9b0 0x7fe93c14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 -- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 msgr2=0x7fe93c0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.161+0000 7fe94119c700 1 -- 172.21.15.53:0/1342784902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe934009a70 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe94119c700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0675d0 tx=0x7fe934004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934004670 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9340047f0 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934012670 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c0d1300 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.162+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0b5520 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.163+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93c0d15e0 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.164+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe934023030 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.166+0000 7fe9397fa700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 0x7fe92006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.166+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe934087090 con 0x7fe93c161340 2023-11-28T11:18:23.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.166+0000 7fe94099b700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 0x7fe92006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.167+0000 7fe94099b700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 0x7fe92006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9300011e0 tx=0x7fe930009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.171+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9340538f0 con 0x7fe93c161340 2023-11-28T11:18:23.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.371+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe93c0cb460 con 0x7fe93c161340 2023-11-28T11:18:23.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.371+0000 7fe9397fa700 1 -- 172.21.15.53:0/1342784902 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe93c0cb460 con 0x7fe93c161340 2023-11-28T11:18:23.373 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:23.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 msgr2=0x7fe92006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 0x7fe92006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9300011e0 tx=0x7fe930009040 comp rx=0 tx=0).stop 2023-11-28T11:18:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 msgr2=0x7fe93c0b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0675d0 tx=0x7fe934004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 shutdown_connections 2023-11-28T11:18:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9200682f0 0x7fe92006a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe93c161340 0x7fe93c0b8480 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe93c0cd7c0 0x7fe93c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 --2- 172.21.15.53:0/1342784902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe93c15c9b0 0x7fe93c14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.373+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 >> 172.21.15.53:0/1342784902 conn(0x7fe93c072600 msgr2=0x7fe93c0b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.374+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 shutdown_connections 2023-11-28T11:18:23.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.374+0000 7fe942bff700 1 -- 172.21.15.53:0/1342784902 wait complete. 2023-11-28T11:18:23.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:23.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.597+0000 7efc25fff700 1 Processor -- start 2023-11-28T11:18:23.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.597+0000 7efc25fff700 1 -- start start 2023-11-28T11:18:23.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.598+0000 7efc25fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc20161340 0x7efc200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc25fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200d1950 con 0x7efc2015c9b0 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc25fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200b5520 con 0x7efc200d2320 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc25fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200b5800 con 0x7efc20161340 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc1f7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efc2015c9b0 0x7efc200b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45330/0 (socket says 172.21.15.53:45330) 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc1f7fe700 1 -- 172.21.15.53:0/60310385 learned_addr learned my addr 172.21.15.53:0/60310385 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc17fff700 1 --2- 172.21.15.53:0/60310385 >> v2:172.21.15.53:3300/0 conn(0x7efc20161340 0x7efc200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc1effd700 1 -- 172.21.15.53:0/60310385 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098537458 0 0) 0x7efc200d1950 con 0x7efc2015c9b0 2023-11-28T11:18:23.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc17fff700 1 -- 172.21.15.53:0/60310385 >> v1:172.21.15.157:6789/0 conn(0x7efc200d2320 legacy=0x7efc200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc17fff700 1 -- 172.21.15.53:0/60310385 >> v1:172.21.15.53:6789/0 conn(0x7efc2015c9b0 legacy=0x7efc200b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.599+0000 7efc17fff700 1 -- 172.21.15.53:0/60310385 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200d1300 con 0x7efc20161340 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc17fff700 1 --2- 172.21.15.53:0/60310385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc20161340 0x7efc200d1de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc08009840 tx=0x7efc08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475475fb56cadb51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc1effd700 1 -- 172.21.15.53:0/60310385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08011830 con 0x7efc20161340 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc1effd700 1 -- 172.21.15.53:0/60310385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08011990 con 0x7efc20161340 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc1effd700 1 -- 172.21.15.53:0/60310385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08011d20 con 0x7efc20161340 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc25fff700 1 -- 172.21.15.53:0/60310385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc20161340 msgr2=0x7efc200d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.600+0000 7efc25fff700 1 --2- 172.21.15.53:0/60310385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc20161340 0x7efc200d1de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc08009840 tx=0x7efc08009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.601+0000 7efc25fff700 1 -- 172.21.15.53:0/60310385 shutdown_connections 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.601+0000 7efc25fff700 1 --2- 172.21.15.53:0/60310385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc20161340 0x7efc200d1de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.601+0000 7efc25fff700 1 -- 172.21.15.53:0/60310385 >> 172.21.15.53:0/60310385 conn(0x7efc20072600 msgr2=0x7efc200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.601+0000 7efc25fff700 1 -- 172.21.15.53:0/60310385 shutdown_connections 2023-11-28T11:18:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.601+0000 7efc25fff700 1 -- 172.21.15.53:0/60310385 wait complete. 2023-11-28T11:18:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.602+0000 7efc25fff700 1 Processor -- start 2023-11-28T11:18:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.602+0000 7efc25fff700 1 -- start start 2023-11-28T11:18:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 0x7efc200ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc20161340 0x7efc201595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200c8600 con 0x7efc200d2320 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200c7cf0 con 0x7efc2015c9b0 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.603+0000 7efc25fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc20148f10 con 0x7efc20161340 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35852/0 (socket says 172.21.15.53:35852) 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 -- 172.21.15.53:0/596442636 learned_addr learned my addr 172.21.15.53:0/596442636 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc17fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 0x7efc200ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1ffff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc20161340 0x7efc201595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 -- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc20161340 msgr2=0x7efc201595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc20161340 0x7efc201595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 -- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 msgr2=0x7efc200ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 0x7efc200ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 -- 172.21.15.53:0/596442636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080094a0 con 0x7efc200d2320 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc17fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 0x7efc200ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1f7fe700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c550 tx=0x7efc1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10004c80 con 0x7efc200d2320 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10004e00 con 0x7efc200d2320 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.604+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2015cd80 con 0x7efc200d2320 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.605+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10007310 con 0x7efc200d2320 2023-11-28T11:18:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.605+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc200d15e0 con 0x7efc200d2320 2023-11-28T11:18:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.606+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2015cf10 con 0x7efc200d2320 2023-11-28T11:18:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.607+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc1001a020 con 0x7efc200d2320 2023-11-28T11:18:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.607+0000 7efc1cff9700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 0x7efc0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.608+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc10085990 con 0x7efc200d2320 2023-11-28T11:18:23.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.608+0000 7efc17fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 0x7efc0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:23.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.608+0000 7efc17fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 0x7efc0006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc080090b0 tx=0x7efc0801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.612+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc10052100 con 0x7efc200d2320 2023-11-28T11:18:23.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.812+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efc200c8740 con 0x7efc200d2320 2023-11-28T11:18:23.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.812+0000 7efc1cff9700 1 -- 172.21.15.53:0/596442636 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7efc10055720 con 0x7efc200d2320 2023-11-28T11:18:23.813 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 msgr2=0x7efc0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 0x7efc0006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc080090b0 tx=0x7efc0801c040 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 msgr2=0x7efc200cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c550 tx=0x7efc1000c910 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 shutdown_connections 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efc000682f0 0x7efc0006a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc20161340 0x7efc201595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc200d2320 0x7efc200cc4a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 --2- 172.21.15.53:0/596442636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2015c9b0 0x7efc200ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.814+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 >> 172.21.15.53:0/596442636 conn(0x7efc20072600 msgr2=0x7efc2015f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:23.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.815+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 shutdown_connections 2023-11-28T11:18:23.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:23.815+0000 7efc25fff700 1 -- 172.21.15.53:0/596442636 wait complete. 2023-11-28T11:18:23.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:24.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.038+0000 7f27a1fff700 1 Processor -- start 2023-11-28T11:18:24.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.038+0000 7f27a1fff700 1 -- start start 2023-11-28T11:18:24.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f27a1fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c15c9b0 0x7f279c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f27a1fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f279c161340 0x7f279c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f27a1fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279c0d1950 con 0x7f279c0d2320 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f27a1fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279c0b5800 con 0x7f279c161340 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f27a1fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0b5520 con 0x7f279c15c9b0 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f279c161340 0x7f279c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f279c161340 0x7f279c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35866/0 (socket says 172.21.15.53:35866) 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 -- 172.21.15.53:0/1380041552 learned_addr learned my addr 172.21.15.53:0/1380041552 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f279b7fe700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c15c9b0 0x7f279c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 -- 172.21.15.53:0/1380041552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c15c9b0 msgr2=0x7f279c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c15c9b0 0x7f279c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 -- 172.21.15.53:0/1380041552 >> v1:172.21.15.157:6789/0 conn(0x7f279c0d2320 legacy=0x7f279c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.040+0000 7f2793fff700 1 -- 172.21.15.53:0/1380041552 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0d1300 con 0x7f279c161340 2023-11-28T11:18:24.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.041+0000 7f2793fff700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c161340 0x7f279c0d1de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27840098f0 tx=0x7f2784009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b0507ee02b0631 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.041+0000 7f279affd700 1 -- 172.21.15.53:0/1380041552 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784011830 con 0x7f279c161340 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.041+0000 7f279affd700 1 -- 172.21.15.53:0/1380041552 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784011990 con 0x7f279c161340 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.041+0000 7f279affd700 1 -- 172.21.15.53:0/1380041552 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784011d20 con 0x7f279c161340 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 -- 172.21.15.53:0/1380041552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c161340 msgr2=0x7f279c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c161340 0x7f279c0d1de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27840098f0 tx=0x7f2784009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:24.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 -- 172.21.15.53:0/1380041552 shutdown_connections 2023-11-28T11:18:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c15c9b0 0x7f279c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 --2- 172.21.15.53:0/1380041552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c161340 0x7f279c0d1de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 -- 172.21.15.53:0/1380041552 >> 172.21.15.53:0/1380041552 conn(0x7f279c072600 msgr2=0x7f279c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 -- 172.21.15.53:0/1380041552 shutdown_connections 2023-11-28T11:18:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.042+0000 7f27a1fff700 1 -- 172.21.15.53:0/1380041552 wait complete. 2023-11-28T11:18:24.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.044+0000 7f27a1fff700 1 Processor -- start 2023-11-28T11:18:24.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.044+0000 7f27a1fff700 1 -- start start 2023-11-28T11:18:24.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0d2320 0x7f279c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c161340 0x7f279c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c1584b0 con 0x7f279c15c9b0 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0c9ad0 con 0x7f279c0d2320 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f27a1fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0c8bc0 con 0x7f279c161340 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35874/0 (socket says 172.21.15.53:35874) 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 -- 172.21.15.53:0/589865954 learned_addr learned my addr 172.21.15.53:0/589865954 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 -- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c161340 msgr2=0x7f279c150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c161340 0x7f279c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 -- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0d2320 msgr2=0x7f279c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0d2320 0x7f279c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.045+0000 7f2793fff700 1 -- 172.21.15.53:0/589865954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784009550 con 0x7f279c15c9b0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f2793fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2784000c00 tx=0x7f2784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784004570 con 0x7f279c15c9b0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278401f070 con 0x7f279c15c9b0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279c0b5130 con 0x7f279c15c9b0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840038c0 con 0x7f279c15c9b0 2023-11-28T11:18:24.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.046+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279c0d15e0 con 0x7f279c15c9b0 2023-11-28T11:18:24.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.047+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279c0b52c0 con 0x7f279c15c9b0 2023-11-28T11:18:24.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.048+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f278402a020 con 0x7f279c15c9b0 2023-11-28T11:18:24.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.049+0000 7f2798ff9700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 0x7f277c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.049+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2784087300 con 0x7f279c15c9b0 2023-11-28T11:18:24.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.049+0000 7f279b7fe700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 0x7f277c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.050+0000 7f279b7fe700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 0x7f277c06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f278c006b20 tx=0x7f278c006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.053+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2784053b60 con 0x7f279c15c9b0 2023-11-28T11:18:24.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.258+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f279c1585f0 con 0x7f279c15c9b0 2023-11-28T11:18:24.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.258+0000 7f2798ff9700 1 -- 172.21.15.53:0/589865954 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f279c1585f0 con 0x7f279c15c9b0 2023-11-28T11:18:24.260 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:24.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 msgr2=0x7f277c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 0x7f277c06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f278c006b20 tx=0x7f278c006ab0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 msgr2=0x7f279c0c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2784000c00 tx=0x7f2784004750 comp rx=0 tx=0).stop 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 shutdown_connections 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f277c0682f0 0x7f277c06a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c161340 0x7f279c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0d2320 0x7f279c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 --2- 172.21.15.53:0/589865954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279c15c9b0 0x7f279c0c6f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 >> 172.21.15.53:0/589865954 conn(0x7f279c072600 msgr2=0x7f279c15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 shutdown_connections 2023-11-28T11:18:24.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.261+0000 7f27a1fff700 1 -- 172.21.15.53:0/589865954 wait complete. 2023-11-28T11:18:24.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:24.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.483+0000 7f5ee93ff700 1 Processor -- start 2023-11-28T11:18:24.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.484+0000 7f5ee93ff700 1 -- start start 2023-11-28T11:18:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee93ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee93ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee40d1950 con 0x7f5ee40d2320 2023-11-28T11:18:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee93ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40b5800 con 0x7f5ee415c9b0 2023-11-28T11:18:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40b5520 con 0x7f5ee4161340 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35876/0 (socket says 172.21.15.53:35876) 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 -- 172.21.15.53:0/3681480013 learned_addr learned my addr 172.21.15.53:0/3681480013 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee27fc700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 -- 172.21.15.53:0/3681480013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 msgr2=0x7f5ee40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.485+0000 7f5ee2ffd700 1 -- 172.21.15.53:0/3681480013 >> v1:172.21.15.157:6789/0 conn(0x7f5ee40d2320 legacy=0x7f5ee40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee2ffd700 1 -- 172.21.15.53:0/3681480013 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40d1300 con 0x7f5ee415c9b0 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee27fc700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098f0 tx=0x7f5ed4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65afd884e0a21459 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee1ffb700 1 -- 172.21.15.53:0/3681480013 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400b8a0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee1ffb700 1 -- 172.21.15.53:0/3681480013 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400ba00 con 0x7f5ee415c9b0 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.486+0000 7f5ee1ffb700 1 -- 172.21.15.53:0/3681480013 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400bd90 con 0x7f5ee415c9b0 2023-11-28T11:18:24.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 -- 172.21.15.53:0/3681480013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 msgr2=0x7f5ee40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098f0 tx=0x7f5ed4009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 -- 172.21.15.53:0/3681480013 shutdown_connections 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee40b4bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/3681480013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 -- 172.21.15.53:0/3681480013 >> 172.21.15.53:0/3681480013 conn(0x7f5ee4072600 msgr2=0x7f5ee40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 -- 172.21.15.53:0/3681480013 shutdown_connections 2023-11-28T11:18:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.487+0000 7f5ee93ff700 1 -- 172.21.15.53:0/3681480013 wait complete. 2023-11-28T11:18:24.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.489+0000 7f5ee93ff700 1 Processor -- start 2023-11-28T11:18:24.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.489+0000 7f5ee93ff700 1 -- start start 2023-11-28T11:18:24.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 0x7f5ee40c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41e9450 con 0x7f5ee415c9b0 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4154c90 con 0x7f5ee40d2320 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4154f70 con 0x7f5ee4161340 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35884/0 (socket says 172.21.15.53:35884) 2023-11-28T11:18:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 -- 172.21.15.53:0/1763914921 learned_addr learned my addr 172.21.15.53:0/1763914921 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 0x7f5ee40c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee37fe700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 -- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 msgr2=0x7f5ee415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee415ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 -- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 msgr2=0x7f5ee40c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 0x7f5ee40c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee27fc700 1 -- 172.21.15.53:0/1763914921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4009550 con 0x7f5ee415c9b0 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.490+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 0x7f5ee40c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ee27fc700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e810 tx=0x7f5ed800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8013610 con 0x7f5ee415c9b0 2023-11-28T11:18:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed8013770 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8013b20 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee40b5130 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.491+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee40d15e0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.492+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee40b52c0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.493+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ed8023070 con 0x7f5ee415c9b0 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.494+0000 7f5ecb7fe700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.494+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ed8085aa0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.494+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.495+0000 7f5ee2ffd700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009c40 tx=0x7f5ed4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.498+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ed8052180 con 0x7f5ee415c9b0 2023-11-28T11:18:24.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.698+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ee40745c0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.699+0000 7f5ecb7fe700 1 -- 172.21.15.53:0/1763914921 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ed80210a0 con 0x7f5ee415c9b0 2023-11-28T11:18:24.701 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:24.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 msgr2=0x7f5ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009c40 tx=0x7f5ed4008030 comp rx=0 tx=0).stop 2023-11-28T11:18:24.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 msgr2=0x7f5ee41e8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e810 tx=0x7f5ed800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 shutdown_connections 2023-11-28T11:18:24.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee4161340 0x7f5ee415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee40d2320 0x7f5ee40c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 --2- 172.21.15.53:0/1763914921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ee415c9b0 0x7f5ee41e8960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 >> 172.21.15.53:0/1763914921 conn(0x7f5ee4072600 msgr2=0x7f5ee4160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 shutdown_connections 2023-11-28T11:18:24.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.701+0000 7f5ee93ff700 1 -- 172.21.15.53:0/1763914921 wait complete. 2023-11-28T11:18:24.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:24.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.923+0000 7ffbe31ff700 1 Processor -- start 2023-11-28T11:18:24.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.923+0000 7ffbe31ff700 1 -- start start 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbe31ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 0x7ffbdc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbe31ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbdc161340 0x7ffbdc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbe31ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc0d1950 con 0x7ffbdc0d2320 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbe31ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc0b5800 con 0x7ffbdc161340 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbe31ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc0b5520 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.924+0000 7ffbdbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbdc161340 0x7ffbdc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbdc161340 0x7ffbdc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35896/0 (socket says 172.21.15.53:35896) 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 -- 172.21.15.53:0/2390813059 learned_addr learned my addr 172.21.15.53:0/2390813059 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 -- 172.21.15.53:0/2390813059 >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 msgr2=0x7ffbdc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/2390813059 >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 0x7ffbdc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 --2- 172.21.15.53:0/2390813059 >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 0x7ffbdc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 -- 172.21.15.53:0/2390813059 >> v1:172.21.15.157:6789/0 conn(0x7ffbdc0d2320 legacy=0x7ffbdc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:24.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 -- 172.21.15.53:0/2390813059 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0d1300 con 0x7ffbdc161340 2023-11-28T11:18:24.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/2390813059 >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 0x7ffbdc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:24.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.925+0000 7ffbdbfff700 1 --2- 172.21.15.53:0/2390813059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc161340 0x7ffbdc0d1de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc067460 tx=0x7ffbc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e894bcc957c3e6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbdb7fe700 1 -- 172.21.15.53:0/2390813059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004230 con 0x7ffbdc161340 2023-11-28T11:18:24.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbdb7fe700 1 -- 172.21.15.53:0/2390813059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8004390 con 0x7ffbdc161340 2023-11-28T11:18:24.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbdb7fe700 1 -- 172.21.15.53:0/2390813059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80036a0 con 0x7ffbdc161340 2023-11-28T11:18:24.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 -- 172.21.15.53:0/2390813059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc161340 msgr2=0x7ffbdc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/2390813059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc161340 0x7ffbdc0d1de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc067460 tx=0x7ffbc8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 -- 172.21.15.53:0/2390813059 shutdown_connections 2023-11-28T11:18:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/2390813059 >> v2:172.21.15.157:3300/0 conn(0x7ffbdc15c9b0 0x7ffbdc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/2390813059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc161340 0x7ffbdc0d1de0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 -- 172.21.15.53:0/2390813059 >> 172.21.15.53:0/2390813059 conn(0x7ffbdc072600 msgr2=0x7ffbdc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 -- 172.21.15.53:0/2390813059 shutdown_connections 2023-11-28T11:18:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.926+0000 7ffbe31ff700 1 -- 172.21.15.53:0/2390813059 wait complete. 2023-11-28T11:18:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.928+0000 7ffbe31ff700 1 Processor -- start 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.928+0000 7ffbe31ff700 1 -- start start 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 0x7ffbdc0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 0x7ffbdc146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0c8840 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0c8b20 con 0x7ffbdc161340 2023-11-28T11:18:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.929+0000 7ffbe31ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0c8e00 con 0x7ffbdc0d2320 2023-11-28T11:18:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35902/0 (socket says 172.21.15.53:35902) 2023-11-28T11:18:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 -- 172.21.15.53:0/376482819 learned_addr learned my addr 172.21.15.53:0/376482819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbe179c700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 0x7ffbdc146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 0x7ffbdc0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 -- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 msgr2=0x7ffbdc0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 0x7ffbdc0c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 -- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 msgr2=0x7ffbdc146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 0x7ffbdc146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 -- 172.21.15.53:0/376482819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009550 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbe179c700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 0x7ffbdc146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 0x7ffbdc0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.930+0000 7ffbdbfff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009130 tx=0x7ffbc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.931+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8003dc0 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.931+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc801f070 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.931+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8011870 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.931+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc0b5130 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.931+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc0d15e0 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.932+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc0b52c0 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.933+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffbc8017020 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.934+0000 7ffbd97fa700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 0x7ffbc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.935+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbc8087620 con 0x7ffbdc15c9b0 2023-11-28T11:18:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.935+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 0x7ffbc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.936+0000 7ffbe0f9b700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 0x7ffbc006a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00089b0 tx=0x7ffbd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:24.939+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffbc8053d90 con 0x7ffbdc15c9b0 2023-11-28T11:18:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.144+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbdc0745c0 con 0x7ffbdc15c9b0 2023-11-28T11:18:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.145+0000 7ffbd97fa700 1 -- 172.21.15.53:0/376482819 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7ffbc80573b0 con 0x7ffbdc15c9b0 2023-11-28T11:18:25.147 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:25.147 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 msgr2=0x7ffbc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 0x7ffbc006a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00089b0 tx=0x7ffbd0008040 comp rx=0 tx=0).stop 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 msgr2=0x7ffbdc150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009130 tx=0x7ffbc8004750 comp rx=0 tx=0).stop 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 shutdown_connections 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffbc00682f0 0x7ffbc006a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbdc0d2320 0x7ffbdc0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc161340 0x7ffbdc146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 --2- 172.21.15.53:0/376482819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbdc15c9b0 0x7ffbdc150830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.147+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 >> 172.21.15.53:0/376482819 conn(0x7ffbdc072600 msgr2=0x7ffbdc15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.148+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 shutdown_connections 2023-11-28T11:18:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.148+0000 7ffbe31ff700 1 -- 172.21.15.53:0/376482819 wait complete. 2023-11-28T11:18:25.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:25.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:25.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.372+0000 7fa1013ff700 1 Processor -- start 2023-11-28T11:18:25.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.372+0000 7fa1013ff700 1 -- start start 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa1013ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0fc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa1013ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc0d1950 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa1013ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc0ce140 con 0x7fa0fc161340 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa1013ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0b5800 con 0x7fa0fc0cd7c0 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa0faffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa0fc15c9b0 0x7fa0fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45342/0 (socket says 172.21.15.53:45342) 2023-11-28T11:18:25.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.373+0000 7fa0faffd700 1 -- 172.21.15.53:0/632338672 learned_addr learned my addr 172.21.15.53:0/632338672 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:25.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0fb7fe700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0fc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858888693 0 0) 0x7fa0fc0d1950 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e40035b0 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140417310 0 0) 0x7fa0e40035b0 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 msgr2=0x7fa0fc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0fc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 >> v1:172.21.15.157:6789/0 conn(0x7fa0fc161340 legacy=0x7fa0fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0d24f0 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa0ec001670 con 0x7fa0fc15c9b0 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 >> v1:172.21.15.53:6789/0 conn(0x7fa0fc15c9b0 legacy=0x7fa0fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4003a30 0x7fa0e4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0b5800 con 0x7fa0e4003a30 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e400bb50 con 0x7fa0e4004d90 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.374+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e400bd10 con 0x7fa0fc0cd7c0 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4003a30 0x7fa0e4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0fa7fc700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0fb7fe700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 -- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 msgr2=0x7fa0e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc0cd7c0 0x7fa0e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 -- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4004d90 msgr2=0x7fa0e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 -- 172.21.15.53:0/632338672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40035b0 con 0x7fa0e4003a30 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0faffd700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4003a30 0x7fa0e4003e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c240 tx=0x7fa0e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1315711f7e5d5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec007c10 con 0x7fa0e4003a30 2023-11-28T11:18:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00ce00 con 0x7fa0e4003a30 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.375+0000 7fa0f9ffb700 1 -- 172.21.15.53:0/632338672 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec005ba0 con 0x7fa0e4003a30 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 -- 172.21.15.53:0/632338672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4003a30 msgr2=0x7fa0e4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 --2- 172.21.15.53:0/632338672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4003a30 0x7fa0e4003e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c240 tx=0x7fa0e4005880 comp rx=0 tx=0).stop 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa0fb7fe700 1 -- 172.21.15.53:0/632338672 reap_dead start 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 -- 172.21.15.53:0/632338672 shutdown_connections 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 -- 172.21.15.53:0/632338672 >> 172.21.15.53:0/632338672 conn(0x7fa0fc072600 msgr2=0x7fa0fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 -- 172.21.15.53:0/632338672 shutdown_connections 2023-11-28T11:18:25.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.376+0000 7fa1013ff700 1 -- 172.21.15.53:0/632338672 wait complete. 2023-11-28T11:18:25.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.378+0000 7fa1013ff700 1 Processor -- start 2023-11-28T11:18:25.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.378+0000 7fa1013ff700 1 -- start start 2023-11-28T11:18:25.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4003a30 0x7fa0fc154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc15e4f0 0x7fa0fc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc15eed0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc1f25a0 con 0x7fa0e4003a30 2023-11-28T11:18:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa1013ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0c89d0 con 0x7fa0fc15e4f0 2023-11-28T11:18:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35930/0 (socket says 172.21.15.53:35930) 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa0fa7fc700 1 -- 172.21.15.53:0/3085689654 learned_addr learned my addr 172.21.15.53:0/3085689654 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa0faffd700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4003a30 0x7fa0fc154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.379+0000 7fa0fb7fe700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc15e4f0 0x7fa0fc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 -- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc15e4f0 msgr2=0x7fa0fc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc15e4f0 0x7fa0fc15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 -- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4003a30 msgr2=0x7fa0fc154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4003a30 0x7fa0fc154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 -- 172.21.15.53:0/3085689654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0d1300 con 0x7fa0fc0c7680 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0fa7fc700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e5b0 tx=0x7fa0f000e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000eee0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f00052e0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0003680 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0b5520 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.380+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc0d15e0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.381+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc1e84a0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.382+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa0f001b030 con 0x7fa0fc0c7680 2023-11-28T11:18:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.383+0000 7fa0e37fe700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 0x7fa0e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.383+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0f00850e0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.384+0000 7fa0faffd700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 0x7fa0e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.384+0000 7fa0faffd700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 0x7fa0e806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec015040 tx=0x7fa0ec007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.387+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0f004d5e0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.588+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0fc0745c0 con 0x7fa0fc0c7680 2023-11-28T11:18:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.588+0000 7fa0e37fe700 1 -- 172.21.15.53:0/3085689654 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0f0019020 con 0x7fa0fc0c7680 2023-11-28T11:18:25.589 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.590+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 msgr2=0x7fa0e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.590+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 0x7fa0e806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec015040 tx=0x7fa0ec007000 comp rx=0 tx=0).stop 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 msgr2=0x7fa0fc0c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e5b0 tx=0x7fa0f000e8c0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 shutdown_connections 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0e80682f0 0x7fa0e806a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0fc15e4f0 0x7fa0fc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0e4003a30 0x7fa0fc154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 --2- 172.21.15.53:0/3085689654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0fc0c7680 0x7fa0fc0c7af0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 >> 172.21.15.53:0/3085689654 conn(0x7fa0fc072600 msgr2=0x7fa0fc0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 shutdown_connections 2023-11-28T11:18:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.591+0000 7fa1013ff700 1 -- 172.21.15.53:0/3085689654 wait complete. 2023-11-28T11:18:25.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.814+0000 7f97465ff700 1 Processor -- start 2023-11-28T11:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.814+0000 7f97465ff700 1 -- start start 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.815+0000 7f97465ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f974015c9b0 0x7f97400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.815+0000 7f97465ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9740161340 0x7f97400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.815+0000 7f97465ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400d1950 con 0x7f97400d2320 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.815+0000 7f97465ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400b5800 con 0x7f9740161340 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.815+0000 7f97465ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400b5520 con 0x7f974015c9b0 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9740161340 0x7f97400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9740161340 0x7f97400d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35938/0 (socket says 172.21.15.53:35938) 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 -- 172.21.15.53:0/2196380304 learned_addr learned my addr 172.21.15.53:0/2196380304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973ffff700 1 --2- 172.21.15.53:0/2196380304 >> v2:172.21.15.157:3300/0 conn(0x7f974015c9b0 0x7f97400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 -- 172.21.15.53:0/2196380304 >> v2:172.21.15.157:3300/0 conn(0x7f974015c9b0 msgr2=0x7f97400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 --2- 172.21.15.53:0/2196380304 >> v2:172.21.15.157:3300/0 conn(0x7f974015c9b0 0x7f97400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 -- 172.21.15.53:0/2196380304 >> v1:172.21.15.157:6789/0 conn(0x7f97400d2320 legacy=0x7f97400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.816+0000 7f973f7fe700 1 -- 172.21.15.53:0/2196380304 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400d1300 con 0x7f9740161340 2023-11-28T11:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f973f7fe700 1 --2- 172.21.15.53:0/2196380304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9740161340 0x7f97400d1de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9740067460 tx=0x7f9734009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3ade6412d31cc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f973effd700 1 -- 172.21.15.53:0/2196380304 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734011830 con 0x7f9740161340 2023-11-28T11:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f973effd700 1 -- 172.21.15.53:0/2196380304 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734011990 con 0x7f9740161340 2023-11-28T11:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f973effd700 1 -- 172.21.15.53:0/2196380304 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734011d20 con 0x7f9740161340 2023-11-28T11:18:25.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 -- 172.21.15.53:0/2196380304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9740161340 msgr2=0x7f97400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 --2- 172.21.15.53:0/2196380304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9740161340 0x7f97400d1de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9740067460 tx=0x7f9734009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 -- 172.21.15.53:0/2196380304 shutdown_connections 2023-11-28T11:18:25.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 --2- 172.21.15.53:0/2196380304 >> v2:172.21.15.157:3300/0 conn(0x7f974015c9b0 0x7f97400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 --2- 172.21.15.53:0/2196380304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9740161340 0x7f97400d1de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.817+0000 7f97465ff700 1 -- 172.21.15.53:0/2196380304 >> 172.21.15.53:0/2196380304 conn(0x7f9740072600 msgr2=0x7f97400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.818+0000 7f97465ff700 1 -- 172.21.15.53:0/2196380304 shutdown_connections 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.818+0000 7f97465ff700 1 -- 172.21.15.53:0/2196380304 wait complete. 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.819+0000 7f97465ff700 1 Processor -- start 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.819+0000 7f97465ff700 1 -- start start 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97400d2320 0x7f9740150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 0x7f974015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740156c20 con 0x7f974015c9b0 2023-11-28T11:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740156010 con 0x7f9740161340 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f97465ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97401562f0 con 0x7f97400d2320 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.820+0000 7f973f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35942/0 (socket says 172.21.15.53:35942) 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 -- 172.21.15.53:0/2002883254 learned_addr learned my addr 172.21.15.53:0/2002883254 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973ffff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97400d2320 0x7f9740150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f9744b9c700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 0x7f974015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 -- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97400d2320 msgr2=0x7f9740150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97400d2320 0x7f9740150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 -- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 msgr2=0x7f974015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 0x7f974015f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 -- 172.21.15.53:0/2002883254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009550 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f9744b9c700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 0x7f974015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.821+0000 7f973f7fe700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9734000c00 tx=0x7f9734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.822+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734004480 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.822+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973401f070 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.822+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340038e0 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.822+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97400b5130 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.822+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97400d15e0 con 0x7f974015c9b0 2023-11-28T11:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.823+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97400b52c0 con 0x7f974015c9b0 2023-11-28T11:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.824+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f973402a020 con 0x7f974015c9b0 2023-11-28T11:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.825+0000 7f973cff9700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 0x7f972c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.825+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97340872a0 con 0x7f974015c9b0 2023-11-28T11:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.825+0000 7f973ffff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 0x7f972c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.826+0000 7f973ffff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 0x7f972c06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9730009be0 tx=0x7f9730009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:25.829+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9734053a10 con 0x7f974015c9b0 2023-11-28T11:18:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.035+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9740156d60 con 0x7f974015c9b0 2023-11-28T11:18:26.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.035+0000 7f973cff9700 1 -- 172.21.15.53:0/2002883254 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9740156d60 con 0x7f974015c9b0 2023-11-28T11:18:26.036 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.037+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 msgr2=0x7f972c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.037+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 0x7f972c06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9730009be0 tx=0x7f9730009210 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.037+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 msgr2=0x7f9740155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.037+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9734000c00 tx=0x7f9734004750 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.037+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 shutdown_connections 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f972c0682f0 0x7f972c06a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97400d2320 0x7f9740150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9740161340 0x7f974015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 --2- 172.21.15.53:0/2002883254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f974015c9b0 0x7f9740155630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 >> 172.21.15.53:0/2002883254 conn(0x7f9740072600 msgr2=0x7f974015a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:26.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 shutdown_connections 2023-11-28T11:18:26.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.038+0000 7f97465ff700 1 -- 172.21.15.53:0/2002883254 wait complete. 2023-11-28T11:18:26.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.261+0000 7f090a7ff700 1 Processor -- start 2023-11-28T11:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.261+0000 7f090a7ff700 1 -- start start 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.262+0000 7f090a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.262+0000 7f090a7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.262+0000 7f090a7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040ce160 con 0x7f0904161340 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.262+0000 7f090a7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040b5800 con 0x7f09040cd7c0 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.262+0000 7f090a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040d1950 con 0x7f090415c9b0 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0908d9c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f09037fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0904161340 0x7f09040d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:34036/0 (socket says 172.21.15.53:34036) 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f09037fe700 1 -- 172.21.15.53:0/845722120 learned_addr learned my addr 172.21.15.53:0/845722120 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0903fff700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0902ffd700 1 -- 172.21.15.53:0/845722120 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169447613 0 0) 0x7f09040ce160 con 0x7f0904161340 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0902ffd700 1 -- 172.21.15.53:0/845722120 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec0035b0 con 0x7f0904161340 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0908d9c700 1 -- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 msgr2=0x7f09040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0908d9c700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.263+0000 7f0908d9c700 1 -- 172.21.15.53:0/845722120 >> v1:172.21.15.157:6789/0 conn(0x7f0904161340 legacy=0x7f09040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0908d9c700 1 -- 172.21.15.53:0/845722120 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040d24f0 con 0x7f09040cd7c0 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0903fff700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0908d9c700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951242836f536c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0902ffd700 1 -- 172.21.15.53:0/845722120 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00e070 con 0x7f09040cd7c0 2023-11-28T11:18:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0902ffd700 1 -- 172.21.15.53:0/845722120 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc00bb20 con 0x7f09040cd7c0 2023-11-28T11:18:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.264+0000 7f0902ffd700 1 -- 172.21.15.53:0/845722120 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc012430 con 0x7f09040cd7c0 2023-11-28T11:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 -- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09040cd7c0 msgr2=0x7f09040cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 -- 172.21.15.53:0/845722120 shutdown_connections 2023-11-28T11:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 --2- 172.21.15.53:0/845722120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 -- 172.21.15.53:0/845722120 >> 172.21.15.53:0/845722120 conn(0x7f0904072600 msgr2=0x7f09040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 -- 172.21.15.53:0/845722120 shutdown_connections 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.265+0000 7f090a7ff700 1 -- 172.21.15.53:0/845722120 wait complete. 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.266+0000 7f090a7ff700 1 Processor -- start 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.267+0000 7f090a7ff700 1 -- start start 2023-11-28T11:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09040cd7c0 0x7f09041f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904161340 0x7f090415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090415f440 con 0x7f090415c9b0 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090415eba0 con 0x7f0904161340 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f090a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090415ee80 con 0x7f09040cd7c0 2023-11-28T11:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35952/0 (socket says 172.21.15.53:35952) 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 -- 172.21.15.53:0/840937520 learned_addr learned my addr 172.21.15.53:0/840937520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f0908d9c700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904161340 0x7f090415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f0903fff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09040cd7c0 0x7f09041f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 -- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09040cd7c0 msgr2=0x7f09041f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09040cd7c0 0x7f09041f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 -- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904161340 msgr2=0x7f090415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904161340 0x7f090415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.268+0000 7f09037fe700 1 -- 172.21.15.53:0/840937520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40094a0 con 0x7f090415c9b0 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f09037fe700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08f8001eb0 tx=0x7f08f800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8005d70 con 0x7f090415c9b0 2023-11-28T11:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f80070d0 con 0x7f090415c9b0 2023-11-28T11:18:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8007460 con 0x7f090415c9b0 2023-11-28T11:18:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc009a70 con 0x7f090415c9b0 2023-11-28T11:18:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.269+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040d1300 con 0x7f090415c9b0 2023-11-28T11:18:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.270+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040b5520 con 0x7f090415c9b0 2023-11-28T11:18:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.271+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08f8019030 con 0x7f090415c9b0 2023-11-28T11:18:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.272+0000 7f0900ff9700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.273+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08f8084ff0 con 0x7f090415c9b0 2023-11-28T11:18:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.273+0000 7f0903fff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.274+0000 7f0903fff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 0x7f08f006a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009cd0 tx=0x7f08f4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.277+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08f8051780 con 0x7f090415c9b0 2023-11-28T11:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.482+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09040d15e0 con 0x7f090415c9b0 2023-11-28T11:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.483+0000 7f0900ff9700 1 -- 172.21.15.53:0/840937520 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08f8017020 con 0x7f090415c9b0 2023-11-28T11:18:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 msgr2=0x7f08f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 0x7f08f006a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009cd0 tx=0x7f08f4009210 comp rx=0 tx=0).stop 2023-11-28T11:18:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 msgr2=0x7f090415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08f8001eb0 tx=0x7f08f800c7b0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 shutdown_connections 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09040cd7c0 0x7f09041f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0904161340 0x7f090415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 --2- 172.21.15.53:0/840937520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f090415c9b0 0x7f090415e2c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 >> 172.21.15.53:0/840937520 conn(0x7f0904072600 msgr2=0x7f09040d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 shutdown_connections 2023-11-28T11:18:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.485+0000 7f090a7ff700 1 -- 172.21.15.53:0/840937520 wait complete. 2023-11-28T11:18:26.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.709+0000 7f063f7ff700 1 Processor -- start 2023-11-28T11:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.709+0000 7f063f7ff700 1 -- start start 2023-11-28T11:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063f7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063f7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380d1950 con 0x7f0638161340 2023-11-28T11:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063f7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380d24f0 con 0x7f06380cd7c0 2023-11-28T11:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063f7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380b5800 con 0x7f063815c9b0 2023-11-28T11:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063cd9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0638161340 0x7f06380d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45354/0 (socket says 172.21.15.53:45354) 2023-11-28T11:18:26.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063cd9a700 1 -- 172.21.15.53:0/3924227238 learned_addr learned my addr 172.21.15.53:0/3924227238 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:26.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.710+0000 7f063d59b700 1 --2- 172.21.15.53:0/3924227238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756235706 0 0) 0x7f06380d1950 con 0x7f0638161340 2023-11-28T11:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06240035b0 con 0x7f0638161340 2023-11-28T11:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409099073 0 0) 0x7f06380d24f0 con 0x7f06380cd7c0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06380d1950 con 0x7f06380cd7c0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f063d59b700 1 -- 172.21.15.53:0/3924227238 >> v1:172.21.15.157:6789/0 conn(0x7f06380cd7c0 legacy=0x7f06380b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f063d59b700 1 -- 172.21.15.53:0/3924227238 >> v1:172.21.15.53:6789/0 conn(0x7f0638161340 legacy=0x7f06380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f063d59b700 1 -- 172.21.15.53:0/3924227238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380b5520 con 0x7f063815c9b0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.711+0000 7f063d59b700 1 --2- 172.21.15.53:0/3924227238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380b4bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0634009870 tx=0x7f0634009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b61405aae2eaaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011830 con 0x7f063815c9b0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0634011990 con 0x7f063815c9b0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f062ffff700 1 -- 172.21.15.53:0/3924227238 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011d20 con 0x7f063815c9b0 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 -- 172.21.15.53:0/3924227238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 msgr2=0x7f06380b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 --2- 172.21.15.53:0/3924227238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380b4bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0634009870 tx=0x7f0634009b80 comp rx=0 tx=0).stop 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 -- 172.21.15.53:0/3924227238 shutdown_connections 2023-11-28T11:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 --2- 172.21.15.53:0/3924227238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380b4bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 -- 172.21.15.53:0/3924227238 >> 172.21.15.53:0/3924227238 conn(0x7f0638072600 msgr2=0x7f06380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 -- 172.21.15.53:0/3924227238 shutdown_connections 2023-11-28T11:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.712+0000 7f063f7ff700 1 -- 172.21.15.53:0/3924227238 wait complete. 2023-11-28T11:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.715+0000 7f063f7ff700 1 Processor -- start 2023-11-28T11:18:26.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.715+0000 7f063f7ff700 1 -- start start 2023-11-28T11:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06380cd7c0 0x7f06380c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381e7490 con 0x7f0638161340 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381e6480 con 0x7f06380cd7c0 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063f7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06381e66d0 con 0x7f063815c9b0 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.716+0000 7f063dd9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35966/0 (socket says 172.21.15.53:35966) 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 -- 172.21.15.53:0/1282969117 learned_addr learned my addr 172.21.15.53:0/1282969117 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063d59b700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06380cd7c0 0x7f06380c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063cd9a700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 -- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 msgr2=0x7f06380c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 -- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06380cd7c0 msgr2=0x7f06380c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06380cd7c0 0x7f06380c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 -- 172.21.15.53:0/1282969117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340094a0 con 0x7f0638161340 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063cd9a700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063dd9c700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f063000cbc0 tx=0x7f063000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630003c00 con 0x7f0638161340 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0630003d60 con 0x7f0638161340 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06380d15e0 con 0x7f0638161340 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.717+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000ab00 con 0x7f0638161340 2023-11-28T11:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.718+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06380d1300 con 0x7f0638161340 2023-11-28T11:18:26.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.719+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380b5130 con 0x7f0638161340 2023-11-28T11:18:26.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.719+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0630010050 con 0x7f0638161340 2023-11-28T11:18:26.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.720+0000 7f062dffb700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 0x7f061c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:26.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.720+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0630085b80 con 0x7f0638161340 2023-11-28T11:18:26.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.721+0000 7f063d59b700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 0x7f061c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:26.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.721+0000 7f063d59b700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 0x7f061c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f063401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:26.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.724+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06300523e0 con 0x7f0638161340 2023-11-28T11:18:26.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.930+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06380745c0 con 0x7f0638161340 2023-11-28T11:18:26.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.930+0000 7f062dffb700 1 -- 172.21.15.53:0/1282969117 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0630055a00 con 0x7f0638161340 2023-11-28T11:18:26.931 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.932+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 msgr2=0x7f061c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.932+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 0x7f061c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f063401c040 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.932+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 msgr2=0x7f063815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.932+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f063000cbc0 tx=0x7f063000ced0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 shutdown_connections 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f061c0682f0 0x7f061c06a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0638161340 0x7f063815a750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06380cd7c0 0x7f06380c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 --2- 172.21.15.53:0/1282969117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f063815c9b0 0x7f06380c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 >> 172.21.15.53:0/1282969117 conn(0x7f0638072600 msgr2=0x7f06380b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 shutdown_connections 2023-11-28T11:18:26.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:26.933+0000 7f063f7ff700 1 -- 172.21.15.53:0/1282969117 wait complete. 2023-11-28T11:18:26.960 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:18:26.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.162+0000 7fac375ff700 1 Processor -- start 2023-11-28T11:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.162+0000 7fac375ff700 1 -- start start 2023-11-28T11:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.163+0000 7fac375ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.163+0000 7fac375ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac300d1950 con 0x7fac3015c9b0 2023-11-28T11:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.163+0000 7fac375ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac300b5520 con 0x7fac300d2320 2023-11-28T11:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.163+0000 7fac375ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac300b5800 con 0x7fac30161340 2023-11-28T11:18:27.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac3539b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fac3015c9b0 0x7fac300b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45360/0 (socket says 172.21.15.53:45360) 2023-11-28T11:18:27.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac3539b700 1 -- 172.21.15.53:0/3087559892 learned_addr learned my addr 172.21.15.53:0/3087559892 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac34b9a700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425772869 0 0) 0x7fac300d1950 con 0x7fac3015c9b0 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac1c0035b0 con 0x7fac3015c9b0 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392626310 0 0) 0x7fac300b5520 con 0x7fac300d2320 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac300d1950 con 0x7fac300d2320 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576803475 0 0) 0x7fac1c0035b0 con 0x7fac3015c9b0 2023-11-28T11:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 msgr2=0x7fac300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 >> v1:172.21.15.157:6789/0 conn(0x7fac300d2320 legacy=0x7fac300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac300d1300 con 0x7fac3015c9b0 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fac2c001670 con 0x7fac3015c9b0 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.164+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 >> v1:172.21.15.53:6789/0 conn(0x7fac3015c9b0 legacy=0x7fac300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1c003bc0 0x7fac1c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac1c004d90 0x7fac1c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300b5800 con 0x7fac1c004d90 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c00bba0 con 0x7fac1c003bc0 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c00bd10 con 0x7fac30161340 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac3539b700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1c003bc0 0x7fac1c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac1c004d90 0x7fac1c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac34b9a700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 -- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 msgr2=0x7fac1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac30161340 0x7fac1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 -- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1c003bc0 msgr2=0x7fac1c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1c003bc0 0x7fac1c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac35b9c700 1 -- 172.21.15.53:0/3087559892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0035b0 con 0x7fac1c004d90 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.165+0000 7fac3539b700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1c003bc0 0x7fac1c003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac35b9c700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac1c004d90 0x7fac1c005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c2f0 tx=0x7fac1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c238fa1b76b71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28001f50 con 0x7fac1c004d90 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac28003a00 con 0x7fac1c004d90 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac27fff700 1 -- 172.21.15.53:0/3087559892 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28003db0 con 0x7fac1c004d90 2023-11-28T11:18:27.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac375ff700 1 -- 172.21.15.53:0/3087559892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac1c004d90 msgr2=0x7fac1c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac375ff700 1 --2- 172.21.15.53:0/3087559892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac1c004d90 0x7fac1c005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c2f0 tx=0x7fac1c005880 comp rx=0 tx=0).stop 2023-11-28T11:18:27.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac35b9c700 1 -- 172.21.15.53:0/3087559892 reap_dead start 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac375ff700 1 -- 172.21.15.53:0/3087559892 shutdown_connections 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.166+0000 7fac375ff700 1 -- 172.21.15.53:0/3087559892 >> 172.21.15.53:0/3087559892 conn(0x7fac30072600 msgr2=0x7fac300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.167+0000 7fac375ff700 1 -- 172.21.15.53:0/3087559892 shutdown_connections 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.167+0000 7fac375ff700 1 -- 172.21.15.53:0/3087559892 wait complete. 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.168+0000 7fac375ff700 1 Processor -- start 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.168+0000 7fac375ff700 1 -- start start 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1c004d90 0x7fac300c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac30160460 0x7fac301608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac30160e10 con 0x7fac301f5d60 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300ca240 con 0x7fac30160460 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.169+0000 7fac375ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac301f6550 con 0x7fac1c004d90 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35986/0 (socket says 172.21.15.53:35986) 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 -- 172.21.15.53:0/4131453884 learned_addr learned my addr 172.21.15.53:0/4131453884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac3539b700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1c004d90 0x7fac300c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac35b9c700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac30160460 0x7fac301608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 -- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1c004d90 msgr2=0x7fac300c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1c004d90 0x7fac300c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 -- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac30160460 msgr2=0x7fac301608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac30160460 0x7fac301608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 -- 172.21.15.53:0/4131453884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3015cd80 con 0x7fac301f5d60 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.170+0000 7fac34b9a700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac20011aa0 tx=0x7fac20011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.171+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000f980 con 0x7fac301f5d60 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.171+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000fb00 con 0x7fac301f5d60 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.171+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac300d15e0 con 0x7fac301f5d60 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.171+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000feb0 con 0x7fac301f5d60 2023-11-28T11:18:27.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.171+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3015cf10 con 0x7fac301f5d60 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.172+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac300745c0 con 0x7fac301f5d60 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.173+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac20014440 con 0x7fac301f5d60 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.174+0000 7fac25ffb700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 0x7fac1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.174+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac200866c0 con 0x7fac301f5d60 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.174+0000 7fac3539b700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 0x7fac1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:27.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.175+0000 7fac3539b700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 0x7fac1406a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac2c001200 tx=0x7fac2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:27.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.178+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac20052f40 con 0x7fac301f5d60 2023-11-28T11:18:27.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.460+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fac300d08a0 con 0x7fac301f5d60 2023-11-28T11:18:27.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.844+0000 7fac25ffb700 1 -- 172.21.15.53:0/4131453884 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v54) v1 ==== 145+0+0 (secure 0 0 0) 0x7fac20056560 con 0x7fac301f5d60 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 msgr2=0x7fac1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 0x7fac1406a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac2c001200 tx=0x7fac2c009040 comp rx=0 tx=0).stop 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 msgr2=0x7fac3014e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac20011aa0 tx=0x7fac20011e60 comp rx=0 tx=0).stop 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 shutdown_connections 2023-11-28T11:18:27.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac140682f0 0x7fac1406a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1c004d90 0x7fac300c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac30160460 0x7fac301608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 --2- 172.21.15.53:0/4131453884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac301f5d60 0x7fac3014e870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 >> 172.21.15.53:0/4131453884 conn(0x7fac30072600 msgr2=0x7fac300b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 shutdown_connections 2023-11-28T11:18:27.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:27.847+0000 7fac375ff700 1 -- 172.21.15.53:0/4131453884 wait complete. 2023-11-28T11:18:27.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:18:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.072+0000 7f0cd95ff700 1 Processor -- start 2023-11-28T11:18:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.072+0000 7f0cd95ff700 1 -- start start 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.073+0000 7f0cd95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.073+0000 7f0cd95ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cd40b16d0 0x7f0cd40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.073+0000 7f0cd95ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40b0230 con 0x7f0cd40b20b0 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.073+0000 7f0cd95ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd415cec0 con 0x7f0cd40b16d0 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.073+0000 7f0cd95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b4570 con 0x7f0cd40d01b0 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cd40b16d0 0x7f0cd40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd27fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0cd40b20b0 0x7f0cd40b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45368/0 (socket says 172.21.15.53:45368) 2023-11-28T11:18:28.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd27fc700 1 -- 172.21.15.53:0/900686263 learned_addr learned my addr 172.21.15.53:0/900686263 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 -- 172.21.15.53:0/900686263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 msgr2=0x7f0cd40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd40d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 -- 172.21.15.53:0/900686263 >> v1:172.21.15.53:6789/0 conn(0x7f0cd40b20b0 legacy=0x7f0cd40b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 -- 172.21.15.53:0/900686263 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40b4850 con 0x7f0cd40b16d0 2023-11-28T11:18:28.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd40d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:28.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.074+0000 7f0cd37fe700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40d1160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009dc0 tx=0x7f0ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb56ad1a81a56683 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd1ffb700 1 -- 172.21.15.53:0/900686263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00e070 con 0x7f0cd40b16d0 2023-11-28T11:18:28.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd1ffb700 1 -- 172.21.15.53:0/900686263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00bb90 con 0x7f0cd40b16d0 2023-11-28T11:18:28.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd1ffb700 1 -- 172.21.15.53:0/900686263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc012430 con 0x7f0cd40b16d0 2023-11-28T11:18:28.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 -- 172.21.15.53:0/900686263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b16d0 msgr2=0x7f0cd40d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40d1160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009dc0 tx=0x7f0ccc00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 -- 172.21.15.53:0/900686263 shutdown_connections 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40d1160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/900686263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd40d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 -- 172.21.15.53:0/900686263 >> 172.21.15.53:0/900686263 conn(0x7f0cd409c3f0 msgr2=0x7f0cd40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 -- 172.21.15.53:0/900686263 shutdown_connections 2023-11-28T11:18:28.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.075+0000 7f0cd95ff700 1 -- 172.21.15.53:0/900686263 wait complete. 2023-11-28T11:18:28.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.077+0000 7f0cd95ff700 1 Processor -- start 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.077+0000 7f0cd95ff700 1 -- start start 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b7640 con 0x7f0cd40b20b0 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b7920 con 0x7f0cd40b16d0 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b7c00 con 0x7f0cd40d01b0 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36008/0 (socket says 172.21.15.53:36008) 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.078+0000 7f0cd27fc700 1 -- 172.21.15.53:0/943915259 learned_addr learned my addr 172.21.15.53:0/943915259 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd37fe700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 -- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 msgr2=0x7f0cd415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd415e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 -- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 msgr2=0x7f0cd40b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 -- 172.21.15.53:0/943915259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009a70 con 0x7f0cd40b20b0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd27fc700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c520 tx=0x7f0cc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8005d70 con 0x7f0cd40b20b0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc80070d0 con 0x7f0cd40b20b0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8007440 con 0x7f0cd40b20b0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40afec0 con 0x7f0cd40b20b0 2023-11-28T11:18:28.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.079+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd40afbe0 con 0x7f0cd40b20b0 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.080+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd40b2600 con 0x7f0cd40b20b0 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.081+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0cc8019030 con 0x7f0cd40b20b0 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.082+0000 7f0cbb7fe700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 0x7f0cc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.082+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cc8084fe0 con 0x7f0cd40b20b0 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.083+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 0x7f0cc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.083+0000 7f0cd2ffd700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 0x7f0cc006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c960 tx=0x7f0cc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.087+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0cc804b5e0 con 0x7f0cd40b20b0 2023-11-28T11:18:28.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.291+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0cd40b8120 con 0x7f0cd40b20b0 2023-11-28T11:18:28.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.295+0000 7f0cbb7fe700 1 -- 172.21.15.53:0/943915259 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0cd40b8120 con 0x7f0cd40b20b0 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 msgr2=0x7f0cc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 0x7f0cc006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c960 tx=0x7f0cc4008040 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 msgr2=0x7f0cd414a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c520 tx=0x7f0cc800c8e0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 shutdown_connections 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0cc00682f0 0x7f0cc006a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cd40d01b0 0x7f0cd415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd40b16d0 0x7f0cd40b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 --2- 172.21.15.53:0/943915259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cd40b20b0 0x7f0cd414a310 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 >> 172.21.15.53:0/943915259 conn(0x7f0cd409c3f0 msgr2=0x7f0cd40b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:28.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 shutdown_connections 2023-11-28T11:18:28.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.298+0000 7f0cd95ff700 1 -- 172.21.15.53:0/943915259 wait complete. 2023-11-28T11:18:28.317 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:18:28.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:18:28.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:18:28.501 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:18:28.501 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:28.501 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:28.545 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:28.545 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:28.629 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:28.629 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:28.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:28.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.955+0000 7fc04e5ff700 1 Processor -- start 2023-11-28T11:18:28.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.955+0000 7fc04e5ff700 1 -- start start 2023-11-28T11:18:28.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.956+0000 7fc04e5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc04815c9b0 0x7fc0480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.956+0000 7fc04e5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc0480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.956+0000 7fc04e5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0480d1950 con 0x7fc0480d2320 2023-11-28T11:18:28.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.956+0000 7fc04e5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0480b5800 con 0x7fc04815c9b0 2023-11-28T11:18:28.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.956+0000 7fc04e5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480b5520 con 0x7fc048161340 2023-11-28T11:18:28.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc04cb9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc0480d2320 0x7fc0480cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45382/0 (socket says 172.21.15.53:45382) 2023-11-28T11:18:28.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc04cb9c700 1 -- 172.21.15.53:0/1847856493 learned_addr learned my addr 172.21.15.53:0/1847856493 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:28.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc0477fe700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc0480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc047fff700 1 --2- 172.21.15.53:0/1847856493 >> v2:172.21.15.157:3300/0 conn(0x7fc04815c9b0 0x7fc0480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427523514 0 0) 0x7fc0480d1950 con 0x7fc0480d2320 2023-11-28T11:18:28.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0300035b0 con 0x7fc0480d2320 2023-11-28T11:18:28.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231449260 0 0) 0x7fc0300035b0 con 0x7fc0480d2320 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 msgr2=0x7fc0480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc0480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 >> v2:172.21.15.157:3300/0 conn(0x7fc04815c9b0 msgr2=0x7fc0480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 --2- 172.21.15.53:0/1847856493 >> v2:172.21.15.157:3300/0 conn(0x7fc04815c9b0 0x7fc0480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480d1300 con 0x7fc0480d2320 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fc0400019f0 con 0x7fc0480d2320 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.957+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 >> v1:172.21.15.53:6789/0 conn(0x7fc0480d2320 legacy=0x7fc0480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc030003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030003f80 0x7fc030004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc030004d90 0x7fc030005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480b5800 con 0x7fc030004d90 2023-11-28T11:18:28.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480b5520 con 0x7fc030003f80 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03000ba40 con 0x7fc048161340 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc030004d90 0x7fc030005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc0477fe700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030003f80 0x7fc030004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc047fff700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc030003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 -- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 msgr2=0x7fc030003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048161340 0x7fc030003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 -- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030003f80 msgr2=0x7fc030004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030003f80 0x7fc030004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.958+0000 7fc04cb9c700 1 -- 172.21.15.53:0/1847856493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300035b0 con 0x7fc030004d90 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc030004d90 0x7fc030005200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc04000a860 tx=0x7fc030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147fb6634742073 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040012600 con 0x7fc030004d90 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040012780 con 0x7fc030004d90 2023-11-28T11:18:28.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc046ffd700 1 -- 172.21.15.53:0/1847856493 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040012b30 con 0x7fc030004d90 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04e5ff700 1 -- 172.21.15.53:0/1847856493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc030004d90 msgr2=0x7fc030005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/1847856493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc030004d90 0x7fc030005200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc04000a860 tx=0x7fc030005880 comp rx=0 tx=0).stop 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04cb9c700 1 -- 172.21.15.53:0/1847856493 reap_dead start 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04e5ff700 1 -- 172.21.15.53:0/1847856493 shutdown_connections 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04e5ff700 1 -- 172.21.15.53:0/1847856493 >> 172.21.15.53:0/1847856493 conn(0x7fc048072600 msgr2=0x7fc0480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.959+0000 7fc04e5ff700 1 -- 172.21.15.53:0/1847856493 shutdown_connections 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.960+0000 7fc04e5ff700 1 -- 172.21.15.53:0/1847856493 wait complete. 2023-11-28T11:18:28.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.962+0000 7fc04e5ff700 1 Processor -- start 2023-11-28T11:18:28.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.962+0000 7fc04e5ff700 1 -- start start 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030004d90 0x7fc0480c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 0x7fc0481e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048152b00 con 0x7fc048152240 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048152de0 con 0x7fc030004d90 2023-11-28T11:18:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.963+0000 7fc04e5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481530c0 con 0x7fc048153ba0 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36022/0 (socket says 172.21.15.53:36022) 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 -- 172.21.15.53:0/2689161748 learned_addr learned my addr 172.21.15.53:0/2689161748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc047fff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030004d90 0x7fc0480c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 0x7fc0481e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 -- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 msgr2=0x7fc0481e85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 0x7fc0481e85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 -- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030004d90 msgr2=0x7fc0480c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030004d90 0x7fc0480c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 -- 172.21.15.53:0/2689161748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480b5130 con 0x7fc048152240 2023-11-28T11:18:28.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc04cb9c700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 0x7fc0481e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:28.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.964+0000 7fc0477fe700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0119a0 tx=0x7fc03c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.965+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011ee0 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.965+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c00f940 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.965+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00fcd0 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.965+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0480d15e0 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.965+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0480b52c0 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.966+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0480745c0 con 0x7fc048152240 2023-11-28T11:18:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.967+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc03c014440 con 0x7fc048152240 2023-11-28T11:18:28.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.968+0000 7fc044ff9700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 0x7fc03406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:28.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.968+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc03c0864f0 con 0x7fc048152240 2023-11-28T11:18:28.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.968+0000 7fc047fff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 0x7fc03406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:28.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.969+0000 7fc047fff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 0x7fc03406a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc038009ce0 tx=0x7fc038009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:28.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:28.972+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc03c052c00 con 0x7fc048152240 2023-11-28T11:18:29.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.175+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0480d08a0 con 0x7fc048152240 2023-11-28T11:18:29.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.177+0000 7fc044ff9700 1 -- 172.21.15.53:0/2689161748 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fc03c056220 con 0x7fc048152240 2023-11-28T11:18:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:29.178 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 msgr2=0x7fc03406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 0x7fc03406a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc038009ce0 tx=0x7fc038009210 comp rx=0 tx=0).stop 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 msgr2=0x7fc0480b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0119a0 tx=0x7fc03c011d60 comp rx=0 tx=0).stop 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 shutdown_connections 2023-11-28T11:18:29.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc0340682f0 0x7fc03406a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048153ba0 0x7fc0481e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030004d90 0x7fc0480c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 --2- 172.21.15.53:0/2689161748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc048152240 0x7fc0480b7300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 >> 172.21.15.53:0/2689161748 conn(0x7fc048072600 msgr2=0x7fc04815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 shutdown_connections 2023-11-28T11:18:29.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.179+0000 7fc04e5ff700 1 -- 172.21.15.53:0/2689161748 wait complete. 2023-11-28T11:18:29.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:29.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:29.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.406+0000 7f343e5ff700 1 Processor -- start 2023-11-28T11:18:29.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.406+0000 7f343e5ff700 1 -- start start 2023-11-28T11:18:29.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343e5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3438161340 0x7f34380d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343e5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34380d2320 0x7f34380cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343e5ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380d1950 con 0x7f343815c9b0 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343e5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380b5800 con 0x7f34380d2320 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343e5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380b5300 con 0x7f3438161340 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343cb9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34380d2320 0x7f34380cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343cb9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34380d2320 0x7f34380cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36032/0 (socket says 172.21.15.53:36032) 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.407+0000 7f343cb9c700 1 -- 172.21.15.53:0/2946696225 learned_addr learned my addr 172.21.15.53:0/2946696225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f34377fe700 1 --2- 172.21.15.53:0/2946696225 >> v2:172.21.15.157:3300/0 conn(0x7f3438161340 0x7f34380d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f343cb9c700 1 -- 172.21.15.53:0/2946696225 >> v2:172.21.15.157:3300/0 conn(0x7f3438161340 msgr2=0x7f34380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f343cb9c700 1 --2- 172.21.15.53:0/2946696225 >> v2:172.21.15.157:3300/0 conn(0x7f3438161340 0x7f34380d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f343cb9c700 1 -- 172.21.15.53:0/2946696225 >> v1:172.21.15.157:6789/0 conn(0x7f343815c9b0 legacy=0x7f34380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f343cb9c700 1 -- 172.21.15.53:0/2946696225 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34380b5520 con 0x7f34380d2320 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f343cb9c700 1 --2- 172.21.15.53:0/2946696225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380d2320 0x7f34380cd7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34380685c0 tx=0x7f343000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216e7645308594b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f3436ffd700 1 -- 172.21.15.53:0/2946696225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000e070 con 0x7f34380d2320 2023-11-28T11:18:29.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f3436ffd700 1 -- 172.21.15.53:0/2946696225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000b980 con 0x7f34380d2320 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.408+0000 7f3436ffd700 1 -- 172.21.15.53:0/2946696225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000bd30 con 0x7f34380d2320 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 -- 172.21.15.53:0/2946696225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380d2320 msgr2=0x7f34380cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 --2- 172.21.15.53:0/2946696225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380d2320 0x7f34380cd7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34380685c0 tx=0x7f343000b330 comp rx=0 tx=0).stop 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 -- 172.21.15.53:0/2946696225 shutdown_connections 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 --2- 172.21.15.53:0/2946696225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380d2320 0x7f34380cd7c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 --2- 172.21.15.53:0/2946696225 >> v2:172.21.15.157:3300/0 conn(0x7f3438161340 0x7f34380d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 -- 172.21.15.53:0/2946696225 >> 172.21.15.53:0/2946696225 conn(0x7f3438072600 msgr2=0x7f34380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:29.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 -- 172.21.15.53:0/2946696225 shutdown_connections 2023-11-28T11:18:29.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.409+0000 7f343e5ff700 1 -- 172.21.15.53:0/2946696225 wait complete. 2023-11-28T11:18:29.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.411+0000 7f343e5ff700 1 Processor -- start 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.412+0000 7f343e5ff700 1 -- start start 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34380d2320 0x7f34380c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f343815c9b0 0x7f3438150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381521e0 con 0x7f3438161340 2023-11-28T11:18:29.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438154dd0 con 0x7f34380d2320 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343e5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3438150c90 con 0x7f343815c9b0 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343cb9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343cb9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36036/0 (socket says 172.21.15.53:36036) 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f343cb9c700 1 -- 172.21.15.53:0/933616929 learned_addr learned my addr 172.21.15.53:0/933616929 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f3437fff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34380d2320 0x7f34380c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.413+0000 7f34377fe700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f343815c9b0 0x7f3438150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 -- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f343815c9b0 msgr2=0x7f3438150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f343815c9b0 0x7f3438150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 -- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34380d2320 msgr2=0x7f34380c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34380d2320 0x7f34380c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 -- 172.21.15.53:0/933616929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009b30 con 0x7f3438161340 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343cb9c700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3430012000 tx=0x7f3430004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300092e0 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3430004960 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34380d15e0 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343001b670 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.414+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34380d1300 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.415+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343815cd80 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.416+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3430021030 con 0x7f3438161340 2023-11-28T11:18:29.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.417+0000 7f3434ff9700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 0x7f342406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.417+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3430087250 con 0x7f3438161340 2023-11-28T11:18:29.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.417+0000 7f3437fff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 0x7f342406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.418+0000 7f3437fff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 0x7f342406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3428001620 tx=0x7f3428009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.421+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34300539c0 con 0x7f3438161340 2023-11-28T11:18:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.624+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34380745c0 con 0x7f3438161340 2023-11-28T11:18:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.624+0000 7f3434ff9700 1 -- 172.21.15.53:0/933616929 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3430056fe0 con 0x7f3438161340 2023-11-28T11:18:29.625 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.626+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 msgr2=0x7f342406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.626+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 0x7f342406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3428001620 tx=0x7f3428009040 comp rx=0 tx=0).stop 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.626+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 msgr2=0x7f34380b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.626+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3430012000 tx=0x7f3430004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 shutdown_connections 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f34240682f0 0x7f342406a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3438161340 0x7f34380b8570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34380d2320 0x7f34380c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 --2- 172.21.15.53:0/933616929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f343815c9b0 0x7f3438150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 >> 172.21.15.53:0/933616929 conn(0x7f3438072600 msgr2=0x7f343815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 shutdown_connections 2023-11-28T11:18:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.627+0000 7f343e5ff700 1 -- 172.21.15.53:0/933616929 wait complete. 2023-11-28T11:18:29.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.850+0000 7f853afff700 1 Processor -- start 2023-11-28T11:18:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.850+0000 7f853afff700 1 -- start start 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853afff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f853415c9b0 0x7f85340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853afff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85340cd7c0 0x7f85340cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853afff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340d1950 con 0x7f8534161340 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853afff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340b5800 con 0x7f85340cd7c0 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853afff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340ce140 con 0x7f853415c9b0 2023-11-28T11:18:29.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85340cd7c0 0x7f85340cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f8538d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f853415c9b0 0x7f85340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85340cd7c0 0x7f85340cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36046/0 (socket says 172.21.15.53:36046) 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 -- 172.21.15.53:0/869072545 learned_addr learned my addr 172.21.15.53:0/869072545 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 -- 172.21.15.53:0/869072545 >> v2:172.21.15.157:3300/0 conn(0x7f853415c9b0 msgr2=0x7f85340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 --2- 172.21.15.53:0/869072545 >> v2:172.21.15.157:3300/0 conn(0x7f853415c9b0 0x7f85340b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 -- 172.21.15.53:0/869072545 >> v1:172.21.15.157:6789/0 conn(0x7f8534161340 legacy=0x7f85340d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.852+0000 7f853959c700 1 -- 172.21.15.53:0/869072545 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340d24f0 con 0x7f85340cd7c0 2023-11-28T11:18:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.853+0000 7f853959c700 1 --2- 172.21.15.53:0/869072545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85340cd7c0 0x7f85340cdc00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8534067460 tx=0x7f852c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826d9696c7bedc55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.853+0000 7f85337fe700 1 -- 172.21.15.53:0/869072545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00e070 con 0x7f85340cd7c0 2023-11-28T11:18:29.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.853+0000 7f85337fe700 1 -- 172.21.15.53:0/869072545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00b910 con 0x7f85340cd7c0 2023-11-28T11:18:29.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.853+0000 7f85337fe700 1 -- 172.21.15.53:0/869072545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00bcc0 con 0x7f85340cd7c0 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 -- 172.21.15.53:0/869072545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85340cd7c0 msgr2=0x7f85340cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 --2- 172.21.15.53:0/869072545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85340cd7c0 0x7f85340cdc00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8534067460 tx=0x7f852c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 -- 172.21.15.53:0/869072545 shutdown_connections 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 --2- 172.21.15.53:0/869072545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85340cd7c0 0x7f85340cdc00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 --2- 172.21.15.53:0/869072545 >> v2:172.21.15.157:3300/0 conn(0x7f853415c9b0 0x7f85340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 -- 172.21.15.53:0/869072545 >> 172.21.15.53:0/869072545 conn(0x7f8534072600 msgr2=0x7f85340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 -- 172.21.15.53:0/869072545 shutdown_connections 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.854+0000 7f853afff700 1 -- 172.21.15.53:0/869072545 wait complete. 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.856+0000 7f853afff700 1 Processor -- start 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.856+0000 7f853afff700 1 -- start start 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85340cd7c0 0x7f853414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415c9b0 0x7f85340b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341e7b50 con 0x7f8534161340 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c79d0 con 0x7f853415c9b0 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853afff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c7cb0 con 0x7f85340cd7c0 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36062/0 (socket says 172.21.15.53:36062) 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 -- 172.21.15.53:0/3907409410 learned_addr learned my addr 172.21.15.53:0/3907409410 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f8533fff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415c9b0 0x7f85340b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f8538d9b700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85340cd7c0 0x7f853414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 -- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85340cd7c0 msgr2=0x7f853414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85340cd7c0 0x7f853414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 -- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415c9b0 msgr2=0x7f85340b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415c9b0 0x7f85340b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.857+0000 7f853959c700 1 -- 172.21.15.53:0/3907409410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009b30 con 0x7f8534161340 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f853959c700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85340675d0 tx=0x7f852c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c004880 con 0x7f8534161340 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c0099b0 con 0x7f8534161340 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340d1300 con 0x7f8534161340 2023-11-28T11:18:29.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c01b670 con 0x7f8534161340 2023-11-28T11:18:29.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.858+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85340b5520 con 0x7f8534161340 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.859+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85340d15e0 con 0x7f8534161340 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.860+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f852c02b070 con 0x7f8534161340 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.861+0000 7f85317fa700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 0x7f851806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.861+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f852c087440 con 0x7f8534161340 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.862+0000 7f8538d9b700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 0x7f851806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.862+0000 7f8538d9b700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 0x7f851806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8528009b40 tx=0x7f8528009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:29.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:29.865+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f852c053bb0 con 0x7f8534161340 2023-11-28T11:18:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.065+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85341e7c90 con 0x7f8534161340 2023-11-28T11:18:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.066+0000 7f85317fa700 1 -- 172.21.15.53:0/3907409410 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85341e7c90 con 0x7f8534161340 2023-11-28T11:18:30.068 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 msgr2=0x7f851806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 0x7f851806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8528009b40 tx=0x7f8528009210 comp rx=0 tx=0).stop 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 msgr2=0x7f85340b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85340675d0 tx=0x7f852c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 shutdown_connections 2023-11-28T11:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f85180682f0 0x7f851806a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85340cd7c0 0x7f853414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8534161340 0x7f85340b7ac0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 --2- 172.21.15.53:0/3907409410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415c9b0 0x7f85340b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 >> 172.21.15.53:0/3907409410 conn(0x7f8534072600 msgr2=0x7f85340b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 shutdown_connections 2023-11-28T11:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.068+0000 7f853afff700 1 -- 172.21.15.53:0/3907409410 wait complete. 2023-11-28T11:18:30.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.290+0000 7f19d37ff700 1 Processor -- start 2023-11-28T11:18:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.290+0000 7f19d37ff700 1 -- start start 2023-11-28T11:18:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d37ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19cc0cd7c0 0x7f19cc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d37ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19cc0d1950 con 0x7f19cc15c9b0 2023-11-28T11:18:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d37ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19cc0ce140 con 0x7f19cc161340 2023-11-28T11:18:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d37ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19cc0b5800 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d159b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19cc15c9b0 0x7f19cc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45394/0 (socket says 172.21.15.53:45394) 2023-11-28T11:18:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.291+0000 7f19d159b700 1 -- 172.21.15.53:0/1954727723 learned_addr learned my addr 172.21.15.53:0/1954727723 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19d1d9c700 1 --2- 172.21.15.53:0/1954727723 >> v2:172.21.15.157:3300/0 conn(0x7f19cc0cd7c0 0x7f19cc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19c3fff700 1 -- 172.21.15.53:0/1954727723 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261413558 0 0) 0x7f19cc0d1950 con 0x7f19cc15c9b0 2023-11-28T11:18:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19c3fff700 1 -- 172.21.15.53:0/1954727723 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19b80035b0 con 0x7f19cc15c9b0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19d1d9c700 1 -- 172.21.15.53:0/1954727723 >> v1:172.21.15.157:6789/0 conn(0x7f19cc161340 legacy=0x7f19cc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19d1d9c700 1 -- 172.21.15.53:0/1954727723 >> v1:172.21.15.53:6789/0 conn(0x7f19cc15c9b0 legacy=0x7f19cc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19d1d9c700 1 -- 172.21.15.53:0/1954727723 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc0d24f0 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.292+0000 7f19d1d9c700 1 --2- 172.21.15.53:0/1954727723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0cdc00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d90 tx=0x7f19c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d2fe748933d232 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19c3fff700 1 -- 172.21.15.53:0/1954727723 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400e070 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19c3fff700 1 -- 172.21.15.53:0/1954727723 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400bb20 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19c3fff700 1 -- 172.21.15.53:0/1954727723 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4012430 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 -- 172.21.15.53:0/1954727723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 msgr2=0x7f19cc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 --2- 172.21.15.53:0/1954727723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0cdc00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d90 tx=0x7f19c400b330 comp rx=0 tx=0).stop 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 -- 172.21.15.53:0/1954727723 shutdown_connections 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 --2- 172.21.15.53:0/1954727723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0cdc00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 -- 172.21.15.53:0/1954727723 >> 172.21.15.53:0/1954727723 conn(0x7f19cc072600 msgr2=0x7f19cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:30.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 -- 172.21.15.53:0/1954727723 shutdown_connections 2023-11-28T11:18:30.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.293+0000 7f19d37ff700 1 -- 172.21.15.53:0/1954727723 wait complete. 2023-11-28T11:18:30.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.295+0000 7f19d37ff700 1 Processor -- start 2023-11-28T11:18:30.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.295+0000 7f19d37ff700 1 -- start start 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 0x7f19cc15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc1f3e30 con 0x7f19cc161340 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc0cd020 con 0x7f19cc0cd7c0 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d37ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc15ece0 con 0x7f19cc15c9b0 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d1d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d1d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36074/0 (socket says 172.21.15.53:36074) 2023-11-28T11:18:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.296+0000 7f19d1d9c700 1 -- 172.21.15.53:0/2180233517 learned_addr learned my addr 172.21.15.53:0/2180233517 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d159b700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d0d9a700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 0x7f19cc15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 -- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 msgr2=0x7f19cc15e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 0x7f19cc15e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 -- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 msgr2=0x7f19cc0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 -- 172.21.15.53:0/2180233517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4009a70 con 0x7f19cc161340 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d0d9a700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 0x7f19cc15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19d1d9c700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0675d0 tx=0x7f19c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4012430 con 0x7f19cc161340 2023-11-28T11:18:30.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.297+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c4020070 con 0x7f19cc161340 2023-11-28T11:18:30.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.298+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc0d1300 con 0x7f19cc161340 2023-11-28T11:18:30.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.298+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4004280 con 0x7f19cc161340 2023-11-28T11:18:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.298+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc0b5520 con 0x7f19cc161340 2023-11-28T11:18:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.299+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc0d15e0 con 0x7f19cc161340 2023-11-28T11:18:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.300+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19c402e070 con 0x7f19cc161340 2023-11-28T11:18:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.300+0000 7f19c1ffb700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.301+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19c4022070 con 0x7f19cc161340 2023-11-28T11:18:30.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.301+0000 7f19d159b700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.302+0000 7f19d159b700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 0x7f19b006a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19c80011e0 tx=0x7f19c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.305+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19c40541a0 con 0x7f19cc161340 2023-11-28T11:18:30.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.505+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19cc1f3f70 con 0x7f19cc161340 2023-11-28T11:18:30.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.505+0000 7f19c1ffb700 1 -- 172.21.15.53:0/2180233517 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19cc1f3f70 con 0x7f19cc161340 2023-11-28T11:18:30.506 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 msgr2=0x7f19b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 0x7f19b006a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19c80011e0 tx=0x7f19c8009040 comp rx=0 tx=0).stop 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 msgr2=0x7f19cc1e6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0675d0 tx=0x7f19c4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 shutdown_connections 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19cc161340 0x7f19cc1e6970 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19cc0cd7c0 0x7f19cc0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 --2- 172.21.15.53:0/2180233517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19cc15c9b0 0x7f19cc15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 >> 172.21.15.53:0/2180233517 conn(0x7f19cc072600 msgr2=0x7f19cc160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:30.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 shutdown_connections 2023-11-28T11:18:30.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.508+0000 7f19d37ff700 1 -- 172.21.15.53:0/2180233517 wait complete. 2023-11-28T11:18:30.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.731+0000 7f10357ff700 1 Processor -- start 2023-11-28T11:18:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.731+0000 7f10357ff700 1 -- start start 2023-11-28T11:18:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.732+0000 7f10357ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f103015c9b0 0x7f10300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.732+0000 7f10357ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10300cd7c0 0x7f10300cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.732+0000 7f10357ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300d1950 con 0x7f1030161340 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.732+0000 7f10357ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300b5800 con 0x7f103015c9b0 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.732+0000 7f10357ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300ce140 con 0x7f10300cd7c0 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f103015c9b0 0x7f10300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f103015c9b0 0x7f10300b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36084/0 (socket says 172.21.15.53:36084) 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 -- 172.21.15.53:0/2775598328 learned_addr learned my addr 172.21.15.53:0/2775598328 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102f7fe700 1 --2- 172.21.15.53:0/2775598328 >> v2:172.21.15.157:3300/0 conn(0x7f10300cd7c0 0x7f10300cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 -- 172.21.15.53:0/2775598328 >> v2:172.21.15.157:3300/0 conn(0x7f10300cd7c0 msgr2=0x7f10300cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 --2- 172.21.15.53:0/2775598328 >> v2:172.21.15.157:3300/0 conn(0x7f10300cd7c0 0x7f10300cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 -- 172.21.15.53:0/2775598328 >> v1:172.21.15.157:6789/0 conn(0x7f1030161340 legacy=0x7f10300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 -- 172.21.15.53:0/2775598328 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300d24f0 con 0x7f103015c9b0 2023-11-28T11:18:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.733+0000 7f102effd700 1 --2- 172.21.15.53:0/2775598328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10300b4bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f103006c170 tx=0x7f1020009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322903887caa95b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f102dffb700 1 -- 172.21.15.53:0/2775598328 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020004230 con 0x7f103015c9b0 2023-11-28T11:18:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f102dffb700 1 -- 172.21.15.53:0/2775598328 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020004390 con 0x7f103015c9b0 2023-11-28T11:18:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f102dffb700 1 -- 172.21.15.53:0/2775598328 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200036a0 con 0x7f103015c9b0 2023-11-28T11:18:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 -- 172.21.15.53:0/2775598328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 msgr2=0x7f10300b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 --2- 172.21.15.53:0/2775598328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10300b4bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f103006c170 tx=0x7f1020009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 -- 172.21.15.53:0/2775598328 shutdown_connections 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 --2- 172.21.15.53:0/2775598328 >> v2:172.21.15.157:3300/0 conn(0x7f10300cd7c0 0x7f10300cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 --2- 172.21.15.53:0/2775598328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10300b4bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 -- 172.21.15.53:0/2775598328 >> 172.21.15.53:0/2775598328 conn(0x7f1030072600 msgr2=0x7f10300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 -- 172.21.15.53:0/2775598328 shutdown_connections 2023-11-28T11:18:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.734+0000 7f10357ff700 1 -- 172.21.15.53:0/2775598328 wait complete. 2023-11-28T11:18:30.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.737+0000 7f10357ff700 1 Processor -- start 2023-11-28T11:18:30.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.737+0000 7f10357ff700 1 -- start start 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10300cd7c0 0x7f10301f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 0x7f10300caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300cbae0 con 0x7f103015c9b0 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1030154cf0 con 0x7f10300cd7c0 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f10357ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1030156660 con 0x7f1030161340 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.738+0000 7f102e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36088/0 (socket says 172.21.15.53:36088) 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 -- 172.21.15.53:0/1386457429 learned_addr learned my addr 172.21.15.53:0/1386457429 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:30.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102effd700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10300cd7c0 0x7f10301f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102f7fe700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 0x7f10300caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 -- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 msgr2=0x7f10300caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 0x7f10300caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 -- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10300cd7c0 msgr2=0x7f10301f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10300cd7c0 0x7f10301f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 -- 172.21.15.53:0/1386457429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020009550 con 0x7f103015c9b0 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102f7fe700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 0x7f10300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:30.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f102e7fc700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f102400b4a0 tx=0x7f102400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024011830 con 0x7f103015c9b0 2023-11-28T11:18:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024011990 con 0x7f103015c9b0 2023-11-28T11:18:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10300d1300 con 0x7f103015c9b0 2023-11-28T11:18:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.739+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024011d40 con 0x7f103015c9b0 2023-11-28T11:18:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.740+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10300b5520 con 0x7f103015c9b0 2023-11-28T11:18:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.741+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10300d15e0 con 0x7f103015c9b0 2023-11-28T11:18:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.741+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f102400f050 con 0x7f103015c9b0 2023-11-28T11:18:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.742+0000 7f10177fe700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 0x7f101c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.742+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1024085860 con 0x7f103015c9b0 2023-11-28T11:18:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.743+0000 7f102effd700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 0x7f101c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.744+0000 7f102effd700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 0x7f101c06a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1020009130 tx=0x7f1020008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:30.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.746+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1024051f70 con 0x7f103015c9b0 2023-11-28T11:18:30.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.949+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10300745c0 con 0x7f103015c9b0 2023-11-28T11:18:30.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.950+0000 7f10177fe700 1 -- 172.21.15.53:0/1386457429 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1024055590 con 0x7f103015c9b0 2023-11-28T11:18:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:30.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 msgr2=0x7f101c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 0x7f101c06a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1020009130 tx=0x7f1020008030 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 msgr2=0x7f10301547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f102400b4a0 tx=0x7f102400b7b0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 shutdown_connections 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f101c0682f0 0x7f101c06a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1030161340 0x7f10300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10300cd7c0 0x7f10301f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 --2- 172.21.15.53:0/1386457429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103015c9b0 0x7f10301547b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 >> 172.21.15.53:0/1386457429 conn(0x7f1030072600 msgr2=0x7f1030159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 shutdown_connections 2023-11-28T11:18:30.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:30.952+0000 7f10357ff700 1 -- 172.21.15.53:0/1386457429 wait complete. 2023-11-28T11:18:30.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:18:31.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.175+0000 7fc8ec5ff700 1 Processor -- start 2023-11-28T11:18:31.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.175+0000 7fc8ec5ff700 1 -- start start 2023-11-28T11:18:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.176+0000 7fc8ec5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc8e4161340 0x7fc8e40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.176+0000 7fc8ec5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8e40d2320 0x7fc8e40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.176+0000 7fc8ec5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40d1950 con 0x7fc8e415c9b0 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.176+0000 7fc8ec5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40b5800 con 0x7fc8e4161340 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.176+0000 7fc8ec5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40b5300 con 0x7fc8e40d2320 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc8e4161340 0x7fc8e40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8ea39b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc8e415c9b0 0x7fc8e40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43236/0 (socket says 172.21.15.53:43236) 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8ea39b700 1 -- 172.21.15.53:0/3622162152 learned_addr learned my addr 172.21.15.53:0/3622162152 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3622162152 >> v2:172.21.15.157:3300/0 conn(0x7fc8e40d2320 msgr2=0x7fc8e40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8eab9c700 1 --2- 172.21.15.53:0/3622162152 >> v2:172.21.15.157:3300/0 conn(0x7fc8e40d2320 0x7fc8e40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9399700 1 -- 172.21.15.53:0/3622162152 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555097429 0 0) 0x7fc8e40d1950 con 0x7fc8e415c9b0 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 --2- 172.21.15.53:0/3622162152 >> v2:172.21.15.157:3300/0 conn(0x7fc8e40d2320 0x7fc8e40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3622162152 >> v1:172.21.15.53:6789/0 conn(0x7fc8e415c9b0 legacy=0x7fc8e40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3622162152 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40b5520 con 0x7fc8e4161340 2023-11-28T11:18:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.177+0000 7fc8e9b9a700 1 --2- 172.21.15.53:0/3622162152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4161340 0x7fc8e40d1de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8e406c170 tx=0x7fc8d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d13b5513d9584e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8e9399700 1 -- 172.21.15.53:0/3622162152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011830 con 0x7fc8e4161340 2023-11-28T11:18:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8e9399700 1 -- 172.21.15.53:0/3622162152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4011990 con 0x7fc8e4161340 2023-11-28T11:18:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8e9399700 1 -- 172.21.15.53:0/3622162152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011d20 con 0x7fc8e4161340 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3622162152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4161340 msgr2=0x7fc8e40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3622162152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4161340 0x7fc8e40d1de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8e406c170 tx=0x7fc8d4009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3622162152 shutdown_connections 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3622162152 >> v2:172.21.15.157:3300/0 conn(0x7fc8e40d2320 0x7fc8e40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3622162152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4161340 0x7fc8e40d1de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3622162152 >> 172.21.15.53:0/3622162152 conn(0x7fc8e4072600 msgr2=0x7fc8e40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3622162152 shutdown_connections 2023-11-28T11:18:31.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.178+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3622162152 wait complete. 2023-11-28T11:18:31.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.180+0000 7fc8ec5ff700 1 Processor -- start 2023-11-28T11:18:31.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.180+0000 7fc8ec5ff700 1 -- start start 2023-11-28T11:18:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.181+0000 7fc8ec5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 0x7fc8e4156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.181+0000 7fc8ec5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.181+0000 7fc8ec5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 0x7fc8e41593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8ec5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4156b90 con 0x7fc8e415c9b0 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8ec5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41e69a0 con 0x7fc8e4161340 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8ec5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41e6c40 con 0x7fc8e40d2320 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53958/0 (socket says 172.21.15.53:53958) 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3517146476 learned_addr learned my addr 172.21.15.53:0/3517146476 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8ea39b700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 0x7fc8e4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 msgr2=0x7fc8e4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8eab9c700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 0x7fc8e41593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 0x7fc8e4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 msgr2=0x7fc8e41593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 0x7fc8e41593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 -- 172.21.15.53:0/3517146476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40094a0 con 0x7fc8e415c9b0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8ea39b700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 0x7fc8e4156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8eab9c700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 0x7fc8e41593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.182+0000 7fc8e9b9a700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40090b0 tx=0x7fc8d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.183+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40038a0 con 0x7fc8e415c9b0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.183+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d401f070 con 0x7fc8e415c9b0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.183+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4003c40 con 0x7fc8e415c9b0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.183+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e40d15e0 con 0x7fc8e415c9b0 2023-11-28T11:18:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.183+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e40d1300 con 0x7fc8e415c9b0 2023-11-28T11:18:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.184+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e415cd80 con 0x7fc8e415c9b0 2023-11-28T11:18:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.185+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8d400f050 con 0x7fc8e415c9b0 2023-11-28T11:18:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.186+0000 7fc8daffd700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 0x7fc8c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.186+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8d4021070 con 0x7fc8e415c9b0 2023-11-28T11:18:31.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.186+0000 7fc8ea39b700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 0x7fc8c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.187+0000 7fc8ea39b700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 0x7fc8c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0007fd0 tx=0x7fc8e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.190+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8d4053ed0 con 0x7fc8e415c9b0 2023-11-28T11:18:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.408+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8e40d2110 con 0x7fc8e415c9b0 2023-11-28T11:18:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.409+0000 7fc8daffd700 1 -- 172.21.15.53:0/3517146476 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc8d4085e60 con 0x7fc8e415c9b0 2023-11-28T11:18:31.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 msgr2=0x7fc8c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 0x7fc8c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0007fd0 tx=0x7fc8e0009040 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 msgr2=0x7fc8e41e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40090b0 tx=0x7fc8d4004750 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 shutdown_connections 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8c80682f0 0x7fc8c806a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e40d2320 0x7fc8e4156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4161340 0x7fc8e41593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 --2- 172.21.15.53:0/3517146476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e415c9b0 0x7fc8e41e6460 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 >> 172.21.15.53:0/3517146476 conn(0x7fc8e4072600 msgr2=0x7fc8e4160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 shutdown_connections 2023-11-28T11:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.411+0000 7fc8ec5ff700 1 -- 172.21.15.53:0/3517146476 wait complete. 2023-11-28T11:18:31.414 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:18:31.431 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:18:31.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:31.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T11:18:31.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:18:31.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:31.516 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:18:31.567 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:31.567 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:31.616 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:31.616 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:31.703 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:31.703 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:31.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.991+0000 7fd6313ff700 1 Processor -- start 2023-11-28T11:18:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.991+0000 7fd6313ff700 1 -- start start 2023-11-28T11:18:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd62c15c9b0 0x7fd62c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c161340 0x7fd62c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd62c0d2320 0x7fd62c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0b5800 con 0x7fd62c0d2320 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0d1950 con 0x7fd62c15c9b0 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.992+0000 7fd6313ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b5520 con 0x7fd62c161340 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd62c0d2320 0x7fd62c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd62c0d2320 0x7fd62c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53960/0 (socket says 172.21.15.53:53960) 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 -- 172.21.15.53:0/2572328750 learned_addr learned my addr 172.21.15.53:0/2572328750 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62affd700 1 --2- 172.21.15.53:0/2572328750 >> v2:172.21.15.157:3300/0 conn(0x7fd62c15c9b0 0x7fd62c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62a7fc700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c161340 0x7fd62c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 -- 172.21.15.53:0/2572328750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c161340 msgr2=0x7fd62c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c161340 0x7fd62c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 -- 172.21.15.53:0/2572328750 >> v2:172.21.15.157:3300/0 conn(0x7fd62c15c9b0 msgr2=0x7fd62c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 --2- 172.21.15.53:0/2572328750 >> v2:172.21.15.157:3300/0 conn(0x7fd62c15c9b0 0x7fd62c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 -- 172.21.15.53:0/2572328750 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0d1300 con 0x7fd62c0d2320 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.993+0000 7fd62b7fe700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c0d2320 0x7fd62c0cd7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd62c067460 tx=0x7fd62400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1eb66b983dd3f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd629ffb700 1 -- 172.21.15.53:0/2572328750 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400e070 con 0x7fd62c0d2320 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd629ffb700 1 -- 172.21.15.53:0/2572328750 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400b9c0 con 0x7fd62c0d2320 2023-11-28T11:18:31.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd629ffb700 1 -- 172.21.15.53:0/2572328750 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400bd70 con 0x7fd62c0d2320 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 -- 172.21.15.53:0/2572328750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c0d2320 msgr2=0x7fd62c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c0d2320 0x7fd62c0cd7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd62c067460 tx=0x7fd62400b330 comp rx=0 tx=0).stop 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 -- 172.21.15.53:0/2572328750 shutdown_connections 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c0d2320 0x7fd62c0cd7c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 --2- 172.21.15.53:0/2572328750 >> v2:172.21.15.157:3300/0 conn(0x7fd62c15c9b0 0x7fd62c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 --2- 172.21.15.53:0/2572328750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c161340 0x7fd62c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 -- 172.21.15.53:0/2572328750 >> 172.21.15.53:0/2572328750 conn(0x7fd62c072600 msgr2=0x7fd62c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 -- 172.21.15.53:0/2572328750 shutdown_connections 2023-11-28T11:18:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.994+0000 7fd6313ff700 1 -- 172.21.15.53:0/2572328750 wait complete. 2023-11-28T11:18:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.996+0000 7fd6313ff700 1 Processor -- start 2023-11-28T11:18:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.996+0000 7fd6313ff700 1 -- start start 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c0d2320 0x7fd62c1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd62c161340 0x7fd62c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c153e30 con 0x7fd62c15c9b0 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c1539e0 con 0x7fd62c161340 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.997+0000 7fd6313ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0c6510 con 0x7fd62c0d2320 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53974/0 (socket says 172.21.15.53:53974) 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 -- 172.21.15.53:0/1339253555 learned_addr learned my addr 172.21.15.53:0/1339253555 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62affd700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c0d2320 0x7fd62c1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62b7fe700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd62c161340 0x7fd62c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 -- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c0d2320 msgr2=0x7fd62c1f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c0d2320 0x7fd62c1f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 -- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd62c161340 msgr2=0x7fd62c0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd62c161340 0x7fd62c0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 -- 172.21.15.53:0/1339253555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624009b30 con 0x7fd62c15c9b0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd62a7fc700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd62000e840 tx=0x7fd62000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.998+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620013610 con 0x7fd62c15c9b0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.999+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd620013770 con 0x7fd62c15c9b0 2023-11-28T11:18:32.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.999+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620013b20 con 0x7fd62c15c9b0 2023-11-28T11:18:32.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.999+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0b5130 con 0x7fd62c15c9b0 2023-11-28T11:18:32.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:31.999+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c0d15e0 con 0x7fd62c15c9b0 2023-11-28T11:18:32.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.000+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c0b52c0 con 0x7fd62c15c9b0 2023-11-28T11:18:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.001+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd620023070 con 0x7fd62c15c9b0 2023-11-28T11:18:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.001+0000 7fd6137fe700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 0x7fd61806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.002+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd620085970 con 0x7fd62c15c9b0 2023-11-28T11:18:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.002+0000 7fd62affd700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 0x7fd61806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.002+0000 7fd62affd700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 0x7fd61806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00b9b0 tx=0x7fd61c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.005+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6200520e0 con 0x7fd62c15c9b0 2023-11-28T11:18:32.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.209+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd62c154f50 con 0x7fd62c15c9b0 2023-11-28T11:18:32.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.210+0000 7fd6137fe700 1 -- 172.21.15.53:0/1339253555 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fd620055700 con 0x7fd62c15c9b0 2023-11-28T11:18:32.212 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:32.212 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:32.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.212+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 msgr2=0x7fd61806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.212+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 0x7fd61806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00b9b0 tx=0x7fd61c008040 comp rx=0 tx=0).stop 2023-11-28T11:18:32.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 msgr2=0x7fd62c1534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd62000e840 tx=0x7fd62000ec00 comp rx=0 tx=0).stop 2023-11-28T11:18:32.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 shutdown_connections 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd6180682f0 0x7fd61806a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd62c0d2320 0x7fd62c1f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd62c161340 0x7fd62c0c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 --2- 172.21.15.53:0/1339253555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd62c15c9b0 0x7fd62c1534a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 >> 172.21.15.53:0/1339253555 conn(0x7fd62c072600 msgr2=0x7fd62c160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 shutdown_connections 2023-11-28T11:18:32.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.213+0000 7fd6313ff700 1 -- 172.21.15.53:0/1339253555 wait complete. 2023-11-28T11:18:32.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:32.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:32.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.437+0000 7f27d7fff700 1 Processor -- start 2023-11-28T11:18:32.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.437+0000 7f27d7fff700 1 -- start start 2023-11-28T11:18:32.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.438+0000 7f27d7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27d00cd7c0 0x7f27d00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.438+0000 7f27d7fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d00d1950 con 0x7f27d015c9b0 2023-11-28T11:18:32.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.438+0000 7f27d7fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d00ce140 con 0x7f27d0161340 2023-11-28T11:18:32.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.438+0000 7f27d7fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d00b5800 con 0x7f27d00cd7c0 2023-11-28T11:18:32.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d5d9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f27d015c9b0 0x7f27d00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43240/0 (socket says 172.21.15.53:43240) 2023-11-28T11:18:32.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d659c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27d00cd7c0 0x7f27d00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d5d9b700 1 -- 172.21.15.53:0/3951496295 learned_addr learned my addr 172.21.15.53:0/3951496295 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:32.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d4d99700 1 -- 172.21.15.53:0/3951496295 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311916488 0 0) 0x7f27d00d1950 con 0x7f27d015c9b0 2023-11-28T11:18:32.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d659c700 1 -- 172.21.15.53:0/3951496295 >> v1:172.21.15.157:6789/0 conn(0x7f27d0161340 legacy=0x7f27d00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d659c700 1 -- 172.21.15.53:0/3951496295 >> v1:172.21.15.53:6789/0 conn(0x7f27d015c9b0 legacy=0x7f27d00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d659c700 1 -- 172.21.15.53:0/3951496295 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00d24f0 con 0x7f27d00cd7c0 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.439+0000 7f27d659c700 1 --2- 172.21.15.53:0/3951496295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d00cd7c0 0x7f27d00cdc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009d90 tx=0x7f27c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a08a67c93dfe7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d4d99700 1 -- 172.21.15.53:0/3951496295 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c800e070 con 0x7f27d00cd7c0 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d4d99700 1 -- 172.21.15.53:0/3951496295 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c800bbd0 con 0x7f27d00cd7c0 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d4d99700 1 -- 172.21.15.53:0/3951496295 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8012430 con 0x7f27d00cd7c0 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 -- 172.21.15.53:0/3951496295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d00cd7c0 msgr2=0x7f27d00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 --2- 172.21.15.53:0/3951496295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d00cd7c0 0x7f27d00cdc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009d90 tx=0x7f27c800b330 comp rx=0 tx=0).stop 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 -- 172.21.15.53:0/3951496295 shutdown_connections 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 --2- 172.21.15.53:0/3951496295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d00cd7c0 0x7f27d00cdc00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 -- 172.21.15.53:0/3951496295 >> 172.21.15.53:0/3951496295 conn(0x7f27d0072600 msgr2=0x7f27d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 -- 172.21.15.53:0/3951496295 shutdown_connections 2023-11-28T11:18:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.440+0000 7f27d7fff700 1 -- 172.21.15.53:0/3951496295 wait complete. 2023-11-28T11:18:32.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.442+0000 7f27d7fff700 1 Processor -- start 2023-11-28T11:18:32.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.442+0000 7f27d7fff700 1 -- start start 2023-11-28T11:18:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.443+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 0x7f27d0159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.443+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 0x7f27d00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d7fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01595d0 con 0x7f27d0161340 2023-11-28T11:18:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d7fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d00c71e0 con 0x7f27d015c9b0 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d7fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01e6bb0 con 0x7f27d00cd7c0 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53982/0 (socket says 172.21.15.53:53982) 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d559a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 0x7f27d00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 -- 172.21.15.53:0/2391295581 learned_addr learned my addr 172.21.15.53:0/2391295581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d5d9b700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 0x7f27d0159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 -- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 msgr2=0x7f27d0159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 0x7f27d0159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 -- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 msgr2=0x7f27d00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 0x7f27d00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d659c700 1 -- 172.21.15.53:0/2391295581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8009a70 con 0x7f27d0161340 2023-11-28T11:18:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d559a700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 0x7f27d00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.444+0000 7f27d5d9b700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 0x7f27d0159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27d659c700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27c8014040 tx=0x7f27c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c801b810 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c8020070 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d00d1300 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c801bbb0 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.445+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d00b5520 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.446+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d00d15e0 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.447+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27c801bd10 con 0x7f27d0161340 2023-11-28T11:18:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.448+0000 7f27c67fc700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.448+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27c8022070 con 0x7f27d0161340 2023-11-28T11:18:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.448+0000 7f27d5d9b700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.449+0000 7f27d5d9b700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 0x7f27b406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000f60 tx=0x7f27cc007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.452+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27c8055030 con 0x7f27d0161340 2023-11-28T11:18:32.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.656+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27d00745c0 con 0x7f27d0161340 2023-11-28T11:18:32.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.656+0000 7f27c67fc700 1 -- 172.21.15.53:0/2391295581 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27c8058650 con 0x7f27d0161340 2023-11-28T11:18:32.658 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:32.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 msgr2=0x7f27b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 0x7f27b406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000f60 tx=0x7f27cc007d00 comp rx=0 tx=0).stop 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 msgr2=0x7f27d01e6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27c8014040 tx=0x7f27c8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 shutdown_connections 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cd7c0 0x7f27d0159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27d0161340 0x7f27d01e6670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 --2- 172.21.15.53:0/2391295581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d015c9b0 0x7f27d00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 >> 172.21.15.53:0/2391295581 conn(0x7f27d0072600 msgr2=0x7f27d00d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:32.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 shutdown_connections 2023-11-28T11:18:32.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.659+0000 7f27d7fff700 1 -- 172.21.15.53:0/2391295581 wait complete. 2023-11-28T11:18:32.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:32.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.884+0000 7feeb59ff700 1 Processor -- start 2023-11-28T11:18:32.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.885+0000 7feeb59ff700 1 -- start start 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeb59ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7feeb00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeb59ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00d1950 con 0x7feeb015c9b0 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeb59ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00ce140 con 0x7feeb0161340 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeb59ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00b5800 con 0x7feeb00cd7c0 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeaeffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feeb015c9b0 0x7feeb00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43252/0 (socket says 172.21.15.53:43252) 2023-11-28T11:18:32.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeaeffd700 1 -- 172.21.15.53:0/3592831697 learned_addr learned my addr 172.21.15.53:0/3592831697 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7feeb00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154105549 0 0) 0x7feeb00d1950 con 0x7feeb015c9b0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.886+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c0035b0 con 0x7feeb015c9b0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477104144 0 0) 0x7feeb00ce140 con 0x7feeb0161340 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb00d1950 con 0x7feeb0161340 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 66210283 0 0) 0x7fee9c0035b0 con 0x7feeb015c9b0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 msgr2=0x7feeb00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7feeb00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 >> v1:172.21.15.157:6789/0 conn(0x7feeb0161340 legacy=0x7feeb00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb00d24f0 con 0x7feeb015c9b0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7feea0001670 con 0x7feeb015c9b0 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7feeb00cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:32.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 >> v1:172.21.15.53:6789/0 conn(0x7feeb015c9b0 legacy=0x7feeb00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7fee9c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee9c004100 0x7fee9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004d90 0x7fee9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00b5800 con 0x7fee9c004100 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00bb50 con 0x7fee9c004d90 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00bd10 con 0x7feeb00cd7c0 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feea7fff700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee9c004100 0x7fee9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeaeffd700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7fee9c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.887+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004d90 0x7fee9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 -- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 msgr2=0x7fee9c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb00cd7c0 0x7fee9c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 -- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004d90 msgr2=0x7fee9c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004d90 0x7fee9c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 -- 172.21.15.53:0/3592831697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009a70 con 0x7fee9c004100 2023-11-28T11:18:32.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004d90 0x7fee9c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:32.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feea7fff700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee9c004100 0x7fee9c004850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee98007fd0 tx=0x7fee9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844fa9ae82ddab34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98001430 con 0x7fee9c004100 2023-11-28T11:18:32.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee980050d0 con 0x7fee9c004100 2023-11-28T11:18:32.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.888+0000 7feeae7fc700 1 -- 172.21.15.53:0/3592831697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98005480 con 0x7fee9c004100 2023-11-28T11:18:32.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 -- 172.21.15.53:0/3592831697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee9c004100 msgr2=0x7fee9c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 --2- 172.21.15.53:0/3592831697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee9c004100 0x7fee9c004850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee98007fd0 tx=0x7fee9c005880 comp rx=0 tx=0).stop 2023-11-28T11:18:32.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeaf7fe700 1 -- 172.21.15.53:0/3592831697 reap_dead start 2023-11-28T11:18:32.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 -- 172.21.15.53:0/3592831697 shutdown_connections 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 -- 172.21.15.53:0/3592831697 >> 172.21.15.53:0/3592831697 conn(0x7feeb0072600 msgr2=0x7feeb00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 -- 172.21.15.53:0/3592831697 shutdown_connections 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.889+0000 7feeb59ff700 1 -- 172.21.15.53:0/3592831697 wait complete. 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.891+0000 7feeb59ff700 1 Processor -- start 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.891+0000 7feeb59ff700 1 -- start start 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004100 0x7feeb015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb01e8f00 0x7feeb00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb015fd70 con 0x7feeb00c90c0 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01e9370 con 0x7fee9c004100 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeb59ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb015f4d0 con 0x7feeb01e8f00 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeaf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeaf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54004/0 (socket says 172.21.15.53:54004) 2023-11-28T11:18:32.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeaf7fe700 1 -- 172.21.15.53:0/1963933590 learned_addr learned my addr 172.21.15.53:0/1963933590 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:32.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feea7fff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb01e8f00 0x7feeb00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.892+0000 7feeaeffd700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004100 0x7feeb015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 -- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb01e8f00 msgr2=0x7feeb00c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb01e8f00 0x7feeb00c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 -- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004100 msgr2=0x7feeb015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004100 0x7feeb015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 -- 172.21.15.53:0/1963933590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb00d1300 con 0x7feeb00c90c0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeaf7fe700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feeb00d2960 tx=0x7feea80114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800e070 con 0x7feeb00c90c0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8003c30 con 0x7feeb00c90c0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb00b5520 con 0x7feeb00c90c0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8003da0 con 0x7feeb00c90c0 2023-11-28T11:18:32.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.893+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb00d15e0 con 0x7feeb00c90c0 2023-11-28T11:18:32.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.894+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb00745c0 con 0x7feeb00c90c0 2023-11-28T11:18:32.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.895+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feea8011a20 con 0x7feeb00c90c0 2023-11-28T11:18:32.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.896+0000 7feeacff9700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 0x7fee9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:32.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.896+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feea8086250 con 0x7feeb00c90c0 2023-11-28T11:18:32.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.897+0000 7feeaeffd700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 0x7fee9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:32.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.897+0000 7feeaeffd700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 0x7fee9006a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feea0000c00 tx=0x7feea0001e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:32.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:32.900+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feea8052960 con 0x7feeb00c90c0 2023-11-28T11:18:33.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.102+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feeb00d0b30 con 0x7feeb00c90c0 2023-11-28T11:18:33.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.102+0000 7feeacff9700 1 -- 172.21.15.53:0/1963933590 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7feea8055f80 con 0x7feeb00c90c0 2023-11-28T11:18:33.103 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:33.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 msgr2=0x7fee9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 0x7fee9006a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feea0000c00 tx=0x7feea0001e00 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 msgr2=0x7feeb00c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feeb00d2960 tx=0x7feea80114b0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 shutdown_connections 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fee900682f0 0x7fee9006a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeb00c90c0 0x7feeb00c9530 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee9c004100 0x7feeb015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 --2- 172.21.15.53:0/1963933590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeb01e8f00 0x7feeb00c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.104+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 >> 172.21.15.53:0/1963933590 conn(0x7feeb0072600 msgr2=0x7feeb00b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.105+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 shutdown_connections 2023-11-28T11:18:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.105+0000 7feeb59ff700 1 -- 172.21.15.53:0/1963933590 wait complete. 2023-11-28T11:18:33.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:33.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.327+0000 7fd860dff700 1 Processor -- start 2023-11-28T11:18:33.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.327+0000 7fd860dff700 1 -- start start 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd860dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd860dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c161340 0x7fd85c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd860dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0d1950 con 0x7fd85c0d2320 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd860dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0b5800 con 0x7fd85c15c9b0 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd860dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0b5520 con 0x7fd85c161340 2023-11-28T11:18:33.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd85a59c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd85a59c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54010/0 (socket says 172.21.15.53:54010) 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.328+0000 7fd85a59c700 1 -- 172.21.15.53:0/1454538771 learned_addr learned my addr 172.21.15.53:0/1454538771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd853fff700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c161340 0x7fd85c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd85a59c700 1 -- 172.21.15.53:0/1454538771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c161340 msgr2=0x7fd85c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd85a59c700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c161340 0x7fd85c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd85a59c700 1 -- 172.21.15.53:0/1454538771 >> v1:172.21.15.157:6789/0 conn(0x7fd85c0d2320 legacy=0x7fd85c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd85a59c700 1 -- 172.21.15.53:0/1454538771 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0d1300 con 0x7fd85c15c9b0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd85a59c700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098f0 tx=0x7fd84c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ba5f5648a9275e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd859d9b700 1 -- 172.21.15.53:0/1454538771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00b8a0 con 0x7fd85c15c9b0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd859d9b700 1 -- 172.21.15.53:0/1454538771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00ba00 con 0x7fd85c15c9b0 2023-11-28T11:18:33.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.329+0000 7fd859d9b700 1 -- 172.21.15.53:0/1454538771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00bd90 con 0x7fd85c15c9b0 2023-11-28T11:18:33.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 -- 172.21.15.53:0/1454538771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 msgr2=0x7fd85c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098f0 tx=0x7fd84c009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:33.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 -- 172.21.15.53:0/1454538771 shutdown_connections 2023-11-28T11:18:33.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c0b4bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 --2- 172.21.15.53:0/1454538771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c161340 0x7fd85c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 -- 172.21.15.53:0/1454538771 >> 172.21.15.53:0/1454538771 conn(0x7fd85c072600 msgr2=0x7fd85c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 -- 172.21.15.53:0/1454538771 shutdown_connections 2023-11-28T11:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.330+0000 7fd860dff700 1 -- 172.21.15.53:0/1454538771 wait complete. 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.332+0000 7fd860dff700 1 Processor -- start 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.332+0000 7fd860dff700 1 -- start start 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c0d2320 0x7fd85c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c161340 0x7fd85c1e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c150760 con 0x7fd85c15c9b0 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c154af0 con 0x7fd85c161340 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd860dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c154dd0 con 0x7fd85c0d2320 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd853fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd853fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54012/0 (socket says 172.21.15.53:54012) 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd853fff700 1 -- 172.21.15.53:0/292616396 learned_addr learned my addr 172.21.15.53:0/292616396 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd85ad9d700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c161340 0x7fd85c1e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.333+0000 7fd85a59c700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c0d2320 0x7fd85c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 -- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c0d2320 msgr2=0x7fd85c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c0d2320 0x7fd85c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 -- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c161340 msgr2=0x7fd85c1e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c161340 0x7fd85c1e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 -- 172.21.15.53:0/292616396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c009550 con 0x7fd85c15c9b0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd853fff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e810 tx=0x7fd84400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400c070 con 0x7fd85c15c9b0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844013610 con 0x7fd85c15c9b0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440139c0 con 0x7fd85c15c9b0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c0b5130 con 0x7fd85c15c9b0 2023-11-28T11:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.334+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0d15e0 con 0x7fd85c15c9b0 2023-11-28T11:18:33.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.335+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c0b52c0 con 0x7fd85c15c9b0 2023-11-28T11:18:33.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.336+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd844023070 con 0x7fd85c15c9b0 2023-11-28T11:18:33.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.337+0000 7fd852ffd700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 0x7fd83c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.337+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd844085b20 con 0x7fd85c15c9b0 2023-11-28T11:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.338+0000 7fd85a59c700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 0x7fd83c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.339+0000 7fd85a59c700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 0x7fd83c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009c40 tx=0x7fd84c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.342+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd844052210 con 0x7fd85c15c9b0 2023-11-28T11:18:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.543+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd85c066d60 con 0x7fd85c15c9b0 2023-11-28T11:18:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.544+0000 7fd852ffd700 1 -- 172.21.15.53:0/292616396 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd84400f040 con 0x7fd85c15c9b0 2023-11-28T11:18:33.545 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 msgr2=0x7fd83c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 0x7fd83c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009c40 tx=0x7fd84c008030 comp rx=0 tx=0).stop 2023-11-28T11:18:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 msgr2=0x7fd85c150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e810 tx=0x7fd84400eb20 comp rx=0 tx=0).stop 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 shutdown_connections 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd83c0682f0 0x7fd83c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85c0d2320 0x7fd85c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85c161340 0x7fd85c1e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 --2- 172.21.15.53:0/292616396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85c15c9b0 0x7fd85c150220 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 >> 172.21.15.53:0/292616396 conn(0x7fd85c072600 msgr2=0x7fd85c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 shutdown_connections 2023-11-28T11:18:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.546+0000 7fd860dff700 1 -- 172.21.15.53:0/292616396 wait complete. 2023-11-28T11:18:33.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.769+0000 7ff25b3ff700 1 Processor -- start 2023-11-28T11:18:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.769+0000 7ff25b3ff700 1 -- start start 2023-11-28T11:18:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25b3ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff25415c9b0 0x7ff2540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25b3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 0x7ff2540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25b3ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540d1950 con 0x7ff2540d2320 2023-11-28T11:18:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25b3ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540b5800 con 0x7ff25415c9b0 2023-11-28T11:18:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25b3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540b5520 con 0x7ff254161340 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25919b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff25415c9b0 0x7ff2540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25899a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 0x7ff2540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25899a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 0x7ff2540d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:57500/0 (socket says 172.21.15.53:57500) 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25899a700 1 -- 172.21.15.53:0/3615185898 learned_addr learned my addr 172.21.15.53:0/3615185898 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25919b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff25415c9b0 0x7ff2540b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:43518/0 (socket says 172.21.15.53:43518) 2023-11-28T11:18:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.770+0000 7ff25999c700 1 --1- 172.21.15.53:0/3615185898 >> v1:172.21.15.157:6789/0 conn(0x7ff2540d2320 0x7ff2540cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:34198/0 (socket says 172.21.15.53:34198) 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff25919b700 1 -- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 msgr2=0x7ff2540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff25919b700 1 --2- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 0x7ff2540d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff25919b700 1 -- 172.21.15.53:0/3615185898 >> v1:172.21.15.157:6789/0 conn(0x7ff2540d2320 legacy=0x7ff2540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff25919b700 1 -- 172.21.15.53:0/3615185898 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540d1300 con 0x7ff25415c9b0 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff25919b700 1 --2- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25415c9b0 0x7ff2540b4bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff250009840 tx=0x7ff250009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344e7b1bc7afa30c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.771+0000 7ff24bfff700 1 -- 172.21.15.53:0/3615185898 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011830 con 0x7ff25415c9b0 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff24bfff700 1 -- 172.21.15.53:0/3615185898 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250011990 con 0x7ff25415c9b0 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff24bfff700 1 -- 172.21.15.53:0/3615185898 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011d20 con 0x7ff25415c9b0 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 -- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25415c9b0 msgr2=0x7ff2540b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25415c9b0 0x7ff2540b4bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff250009840 tx=0x7ff250009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 -- 172.21.15.53:0/3615185898 shutdown_connections 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25415c9b0 0x7ff2540b4bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/3615185898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff254161340 0x7ff2540d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 -- 172.21.15.53:0/3615185898 >> 172.21.15.53:0/3615185898 conn(0x7ff254072600 msgr2=0x7ff2540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 -- 172.21.15.53:0/3615185898 shutdown_connections 2023-11-28T11:18:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.772+0000 7ff25b3ff700 1 -- 172.21.15.53:0/3615185898 wait complete. 2023-11-28T11:18:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.774+0000 7ff25b3ff700 1 Processor -- start 2023-11-28T11:18:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.774+0000 7ff25b3ff700 1 -- start start 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 0x7ff2540c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254161340 0x7ff25415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c6410 con 0x7ff25415c9b0 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff254151af0 con 0x7ff254161340 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25b3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff254151dd0 con 0x7ff2540d2320 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25899a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25899a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54016/0 (socket says 172.21.15.53:54016) 2023-11-28T11:18:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25899a700 1 -- 172.21.15.53:0/4169494097 learned_addr learned my addr 172.21.15.53:0/4169494097 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.775+0000 7ff25919b700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 0x7ff2540c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25999c700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254161340 0x7ff25415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 -- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 msgr2=0x7ff2540c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 0x7ff2540c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 -- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254161340 msgr2=0x7ff25415ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254161340 0x7ff25415ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 -- 172.21.15.53:0/4169494097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2500094a0 con 0x7ff25415c9b0 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25919b700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 0x7ff2540c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff25899a700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e910 tx=0x7ff24400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.776+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440096a0 con 0x7ff25415c9b0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.777+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244013680 con 0x7ff25415c9b0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.777+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244013a30 con 0x7ff25415c9b0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.777+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540b5130 con 0x7ff25415c9b0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.777+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2540d15e0 con 0x7ff25415c9b0 2023-11-28T11:18:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.778+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2540b52c0 con 0x7ff25415c9b0 2023-11-28T11:18:33.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.779+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff244019030 con 0x7ff25415c9b0 2023-11-28T11:18:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.780+0000 7ff249ffb700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 0x7ff23806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.780+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff244085b50 con 0x7ff25415c9b0 2023-11-28T11:18:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.780+0000 7ff25919b700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 0x7ff23806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:33.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.781+0000 7ff25919b700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 0x7ff23806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff250000c00 tx=0x7ff250008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.784+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2440522c0 con 0x7ff25415c9b0 2023-11-28T11:18:33.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.988+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff254067b10 con 0x7ff25415c9b0 2023-11-28T11:18:33.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.988+0000 7ff249ffb700 1 -- 172.21.15.53:0/4169494097 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2440558e0 con 0x7ff25415c9b0 2023-11-28T11:18:33.989 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:33.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 msgr2=0x7ff23806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 0x7ff23806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff250000c00 tx=0x7ff250008030 comp rx=0 tx=0).stop 2023-11-28T11:18:33.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 msgr2=0x7ff2541577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e910 tx=0x7ff24400ec20 comp rx=0 tx=0).stop 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 shutdown_connections 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2380682f0 0x7ff23806a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2540d2320 0x7ff2540c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254161340 0x7ff25415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 --2- 172.21.15.53:0/4169494097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff25415c9b0 0x7ff2541577e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.990+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 >> 172.21.15.53:0/4169494097 conn(0x7ff254072600 msgr2=0x7ff25406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.991+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 shutdown_connections 2023-11-28T11:18:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:33.991+0000 7ff25b3ff700 1 -- 172.21.15.53:0/4169494097 wait complete. 2023-11-28T11:18:34.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.214+0000 7f351e1ff700 1 Processor -- start 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.214+0000 7f351e1ff700 1 -- start start 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3518158200 0x7f3518070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35180caa70 0x7f35180b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35180caf10 0x7f35180b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351815f070 con 0x7f35180caf10 2023-11-28T11:18:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518071190 con 0x7f35180caa70 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f351e1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351815e1a0 con 0x7f3518158200 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f3517fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35180caf10 0x7f35180b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f3517fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35180caf10 0x7f35180b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54028/0 (socket says 172.21.15.53:54028) 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.215+0000 7f3517fff700 1 -- 172.21.15.53:0/1874938235 learned_addr learned my addr 172.21.15.53:0/1874938235 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f35177fe700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3518158200 0x7f3518070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3516ffd700 1 --2- 172.21.15.53:0/1874938235 >> v2:172.21.15.157:3300/0 conn(0x7f35180caa70 0x7f35180b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 -- 172.21.15.53:0/1874938235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3518158200 msgr2=0x7f3518070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3518158200 0x7f3518070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 -- 172.21.15.53:0/1874938235 >> v2:172.21.15.157:3300/0 conn(0x7f35180caa70 msgr2=0x7f35180b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 --2- 172.21.15.53:0/1874938235 >> v2:172.21.15.157:3300/0 conn(0x7f35180caa70 0x7f35180b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 -- 172.21.15.53:0/1874938235 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35181585d0 con 0x7f35180caf10 2023-11-28T11:18:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f3517fff700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caf10 0x7f35180b0bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35180672e0 tx=0x7f351000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5771f9d319ca9049 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f35167fc700 1 -- 172.21.15.53:0/1874938235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000e070 con 0x7f35180caf10 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f35167fc700 1 -- 172.21.15.53:0/1874938235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000b8d0 con 0x7f35180caf10 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.216+0000 7f35167fc700 1 -- 172.21.15.53:0/1874938235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000bc80 con 0x7f35180caf10 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 -- 172.21.15.53:0/1874938235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caf10 msgr2=0x7f35180b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caf10 0x7f35180b0bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35180672e0 tx=0x7f351000b330 comp rx=0 tx=0).stop 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 -- 172.21.15.53:0/1874938235 shutdown_connections 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3518158200 0x7f3518070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 --2- 172.21.15.53:0/1874938235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caf10 0x7f35180b0bb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 --2- 172.21.15.53:0/1874938235 >> v2:172.21.15.157:3300/0 conn(0x7f35180caa70 0x7f35180b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 -- 172.21.15.53:0/1874938235 >> 172.21.15.53:0/1874938235 conn(0x7f35180cb670 msgr2=0x7f35180cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 -- 172.21.15.53:0/1874938235 shutdown_connections 2023-11-28T11:18:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.217+0000 7f351e1ff700 1 -- 172.21.15.53:0/1874938235 wait complete. 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.219+0000 7f351e1ff700 1 Processor -- start 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.219+0000 7f351e1ff700 1 -- start start 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35180caf10 0x7f35180cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3518158200 0x7f35181567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180ad640 con 0x7f35180caa70 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180ad920 con 0x7f3518158200 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f351e1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180adc00 con 0x7f35180caf10 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f35177fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f35177fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54042/0 (socket says 172.21.15.53:54042) 2023-11-28T11:18:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.220+0000 7f35177fe700 1 -- 172.21.15.53:0/3477484687 learned_addr learned my addr 172.21.15.53:0/3477484687 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f3516ffd700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35180caf10 0x7f35180cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f3517fff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3518158200 0x7f35181567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 -- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35180caf10 msgr2=0x7f35180cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35180caf10 0x7f35180cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 -- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3518158200 msgr2=0x7f35181567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3518158200 0x7f35181567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 -- 172.21.15.53:0/3477484687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009b30 con 0x7f35180caa70 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f35177fe700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f350800e870 tx=0x7f350800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508009590 con 0x7f35180caa70 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800ee70 con 0x7f35180caa70 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351815e6d0 con 0x7f35180caa70 2023-11-28T11:18:34.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800ca50 con 0x7f35180caa70 2023-11-28T11:18:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.221+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351815e3f0 con 0x7f35180caa70 2023-11-28T11:18:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.223+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351815e9b0 con 0x7f35180caa70 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.224+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3508023070 con 0x7f35180caa70 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.224+0000 7f34fffff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.224+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3508085ee0 con 0x7f35180caa70 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.225+0000 7f3516ffd700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.226+0000 7f3516ffd700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 0x7f350406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35180c9c20 tx=0x7f350c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.228+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3508052650 con 0x7f35180caa70 2023-11-28T11:18:34.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.446+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f351806fd60 con 0x7f35180caa70 2023-11-28T11:18:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.447+0000 7f34fffff700 1 -- 172.21.15.53:0/3477484687 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3508055c70 con 0x7f35180caa70 2023-11-28T11:18:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 msgr2=0x7f350406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 0x7f350406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35180c9c20 tx=0x7f350c008040 comp rx=0 tx=0).stop 2023-11-28T11:18:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 msgr2=0x7f35181f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f350800e870 tx=0x7f350800ec30 comp rx=0 tx=0).stop 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 shutdown_connections 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3518158200 0x7f35181567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35180caa70 0x7f35181f5cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 --2- 172.21.15.53:0/3477484687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35180caf10 0x7f35180cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 >> 172.21.15.53:0/3477484687 conn(0x7f35180cb670 msgr2=0x7f35180ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 shutdown_connections 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:34.449+0000 7f351e1ff700 1 -- 172.21.15.53:0/3477484687 wait complete. 2023-11-28T11:18:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:18:34.468 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T11:18:34.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:34.468 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T11:18:34.565 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:18:34.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:34.566 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:18:34.621 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:34.622 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:34.668 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:34.669 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:34.753 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:18:34.753 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:34.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.078+0000 7f91d5bff700 1 Processor -- start 2023-11-28T11:18:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.078+0000 7f91d5bff700 1 -- start start 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91d5bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0159860 0x7f91d00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91d5bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91d00b0d80 0x7f91d00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91d5bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d00ccc00 con 0x7f91d0159490 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91d5bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00ccec0 con 0x7f91d00b0d80 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91d5bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00cd1a0 con 0x7f91d0159860 2023-11-28T11:18:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91cf7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f91d0159490 0x7f91d00b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43260/0 (socket says 172.21.15.53:43260) 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.079+0000 7f91cf7fe700 1 -- 172.21.15.53:0/3161190143 learned_addr learned my addr 172.21.15.53:0/3161190143 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 --2- 172.21.15.53:0/3161190143 >> v2:172.21.15.53:3300/0 conn(0x7f91d00b0d80 0x7f91d00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91ceffd700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0159860 0x7f91d00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91ce7fc700 1 -- 172.21.15.53:0/3161190143 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750090288 0 0) 0x7f91d00ccc00 con 0x7f91d0159490 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91ce7fc700 1 -- 172.21.15.53:0/3161190143 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b80035b0 con 0x7f91d0159490 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 -- 172.21.15.53:0/3161190143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0159860 msgr2=0x7f91d00c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0159860 0x7f91d00c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 -- 172.21.15.53:0/3161190143 >> v1:172.21.15.53:6789/0 conn(0x7f91d0159490 legacy=0x7f91d00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 -- 172.21.15.53:0/3161190143 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00cd480 con 0x7f91d00b0d80 2023-11-28T11:18:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.080+0000 7f91cffff700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d00b0d80 0x7f91d00c9a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009d90 tx=0x7f91c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0088195bd02b2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91ce7fc700 1 -- 172.21.15.53:0/3161190143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800e070 con 0x7f91d00b0d80 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91ce7fc700 1 -- 172.21.15.53:0/3161190143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800bb20 con 0x7f91d00b0d80 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91ce7fc700 1 -- 172.21.15.53:0/3161190143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800be90 con 0x7f91d00b0d80 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 -- 172.21.15.53:0/3161190143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d00b0d80 msgr2=0x7f91d00c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d00b0d80 0x7f91d00c9a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009d90 tx=0x7f91c800b330 comp rx=0 tx=0).stop 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 -- 172.21.15.53:0/3161190143 shutdown_connections 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d00b0d80 0x7f91d00c9a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3161190143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0159860 0x7f91d00c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 -- 172.21.15.53:0/3161190143 >> 172.21.15.53:0/3161190143 conn(0x7f91d00cb670 msgr2=0x7f91d00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 -- 172.21.15.53:0/3161190143 shutdown_connections 2023-11-28T11:18:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.081+0000 7f91d5bff700 1 -- 172.21.15.53:0/3161190143 wait complete. 2023-11-28T11:18:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.083+0000 7f91d5bff700 1 Processor -- start 2023-11-28T11:18:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.083+0000 7f91d5bff700 1 -- start start 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d00b0d80 0x7f91d0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0153d70 0x7f91d01531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d01545f0 con 0x7f91d0159490 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b47c0 con 0x7f91d00b0d80 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91d5bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0155fa0 con 0x7f91d0153d70 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91ceffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91ceffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54064/0 (socket says 172.21.15.53:54064) 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.084+0000 7f91ceffd700 1 -- 172.21.15.53:0/3122471925 learned_addr learned my addr 172.21.15.53:0/3122471925 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 -- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0153d70 msgr2=0x7f91d01531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0153d70 0x7f91d01531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 -- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d00b0d80 msgr2=0x7f91d0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d00b0d80 0x7f91d0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 -- 172.21.15.53:0/3122471925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009a70 con 0x7f91d0159490 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91ceffd700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e8a0 tx=0x7f91c400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009590 con 0x7f91d0159490 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c400eea0 con 0x7f91d0159490 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c400ca50 con 0x7f91d0159490 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d00c9f40 con 0x7f91d0159490 2023-11-28T11:18:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.085+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d00cd760 con 0x7f91d0159490 2023-11-28T11:18:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.086+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d00cdc80 con 0x7f91d0159490 2023-11-28T11:18:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.087+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91c4023070 con 0x7f91d0159490 2023-11-28T11:18:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.088+0000 7f91b7fff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 0x7f91bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.088+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91c4085ec0 con 0x7f91d0159490 2023-11-28T11:18:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.089+0000 7f91cf7fe700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 0x7f91bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.090+0000 7f91cf7fe700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 0x7f91bc06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91c0007fd0 tx=0x7f91c0007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.092+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91c40525d0 con 0x7f91d0159490 2023-11-28T11:18:35.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.297+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d01561d0 con 0x7f91d0159490 2023-11-28T11:18:35.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.298+0000 7f91b7fff700 1 -- 172.21.15.53:0/3122471925 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f91c4055bf0 con 0x7f91d0159490 2023-11-28T11:18:35.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:35.299 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:17:50.191030+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.300+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 msgr2=0x7f91bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.300+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 0x7f91bc06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91c0007fd0 tx=0x7f91c0007980 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.300+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 msgr2=0x7f91d0157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.300+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e8a0 tx=0x7f91c400ec60 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 shutdown_connections 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f91bc0682f0 0x7f91bc06a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91d0153d70 0x7f91d01531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91d00b0d80 0x7f91d0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 --2- 172.21.15.53:0/3122471925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91d0159490 0x7f91d0157a40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 >> 172.21.15.53:0/3122471925 conn(0x7f91d00cb670 msgr2=0x7f91d015d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:35.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 shutdown_connections 2023-11-28T11:18:35.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.301+0000 7f91d5bff700 1 -- 172.21.15.53:0/3122471925 wait complete. 2023-11-28T11:18:35.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:35.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.523+0000 7fc9007ff700 1 Processor -- start 2023-11-28T11:18:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.523+0000 7fc9007ff700 1 -- start start 2023-11-28T11:18:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.524+0000 7fc9007ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8f80cd7c0 0x7fc8f80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.524+0000 7fc9007ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f80d1950 con 0x7fc8f815c9b0 2023-11-28T11:18:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.524+0000 7fc9007ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f80ce140 con 0x7fc8f8161340 2023-11-28T11:18:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.524+0000 7fc9007ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f80b5800 con 0x7fc8f80cd7c0 2023-11-28T11:18:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fe59b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc8f815c9b0 0x7fc8f80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43268/0 (socket says 172.21.15.53:43268) 2023-11-28T11:18:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fe59b700 1 -- 172.21.15.53:0/2645991330 learned_addr learned my addr 172.21.15.53:0/2645991330 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fed9c700 1 --2- 172.21.15.53:0/2645991330 >> v2:172.21.15.157:3300/0 conn(0x7fc8f80cd7c0 0x7fc8f80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022231482 0 0) 0x7fc8f80d1950 con 0x7fc8f815c9b0 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8e40035b0 con 0x7fc8f815c9b0 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712480498 0 0) 0x7fc8e40035b0 con 0x7fc8f815c9b0 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 >> v2:172.21.15.157:3300/0 conn(0x7fc8f80cd7c0 msgr2=0x7fc8f80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 --2- 172.21.15.53:0/2645991330 >> v2:172.21.15.157:3300/0 conn(0x7fc8f80cd7c0 0x7fc8f80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 >> v1:172.21.15.157:6789/0 conn(0x7fc8f8161340 legacy=0x7fc8f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.525+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80d24f0 con 0x7fc8f815c9b0 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fc8f4001670 con 0x7fc8f815c9b0 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 >> v1:172.21.15.53:6789/0 conn(0x7fc8f815c9b0 legacy=0x7fc8f80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4003a00 0x7fc8e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 0x7fc8e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4005200 0x7fc8e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80b5800 con 0x7fc8e4004850 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e400c110 con 0x7fc8e4003a00 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e400c2d0 con 0x7fc8e4005200 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 0x7fc8e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fe59b700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4003a00 0x7fc8e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fed9c700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4005200 0x7fc8e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4005200 msgr2=0x7fc8e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4005200 0x7fc8e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4003a00 msgr2=0x7fc8e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4003a00 0x7fc8e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/2645991330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40035b0 con 0x7fc8e4004850 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fe59b700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e4003a00 0x7fc8e4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.526+0000 7fc8fed9c700 1 -- 172.21.15.53:0/2645991330 reap_dead start 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 0x7fc8e4004cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80017c0 tx=0x7fc8e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5305fb940f79d73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80052d0 con 0x7fc8e4004850 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e8005430 con 0x7fc8e4004850 2023-11-28T11:18:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc8fd599700 1 -- 172.21.15.53:0/2645991330 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80038c0 con 0x7fc8e4004850 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 -- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 msgr2=0x7fc8e4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 0x7fc8e4004cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80017c0 tx=0x7fc8e4005880 comp rx=0 tx=0).stop 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 -- 172.21.15.53:0/2645991330 shutdown_connections 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 --2- 172.21.15.53:0/2645991330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4004850 0x7fc8e4004cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 -- 172.21.15.53:0/2645991330 >> 172.21.15.53:0/2645991330 conn(0x7fc8f8072600 msgr2=0x7fc8f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 -- 172.21.15.53:0/2645991330 shutdown_connections 2023-11-28T11:18:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.527+0000 7fc9007ff700 1 -- 172.21.15.53:0/2645991330 wait complete. 2023-11-28T11:18:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.529+0000 7fc9007ff700 1 Processor -- start 2023-11-28T11:18:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.529+0000 7fc9007ff700 1 -- start start 2023-11-28T11:18:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4004850 0x7fc8f81e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8f8147340 0x7fc8f81477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8159cb0 con 0x7fc8f80ca360 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8146330 con 0x7fc8f8147340 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.530+0000 7fc9007ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8146580 con 0x7fc8e4004850 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54084/0 (socket says 172.21.15.53:54084) 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/4120385142 learned_addr learned my addr 172.21.15.53:0/4120385142 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4004850 msgr2=0x7fc8f81e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4004850 0x7fc8f81e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8f8147340 msgr2=0x7fc8f81477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8f8147340 0x7fc8f81477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 -- 172.21.15.53:0/4120385142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80d1300 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8fdd9a700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c0e0 tx=0x7fc8e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.531+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8001290 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.532+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e801d480 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.532+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e801d830 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.532+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f80b5520 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.532+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f80d15e0 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.533+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f80745c0 con 0x7fc8f80ca360 2023-11-28T11:18:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.533+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8e802f070 con 0x7fc8f80ca360 2023-11-28T11:18:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.534+0000 7fc8eeffd700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 0x7fc8dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.534+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8e8086a50 con 0x7fc8f80ca360 2023-11-28T11:18:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.535+0000 7fc8fe59b700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 0x7fc8dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.535+0000 7fc8fe59b700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 0x7fc8dc06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4001bb0 tx=0x7fc8f4001110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.538+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8e80531c0 con 0x7fc8f80ca360 2023-11-28T11:18:35.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.738+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8f80d08a0 con 0x7fc8f80ca360 2023-11-28T11:18:35.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.738+0000 7fc8eeffd700 1 -- 172.21.15.53:0/4120385142 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8e802d0a0 con 0x7fc8f80ca360 2023-11-28T11:18:35.740 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 msgr2=0x7fc8dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 0x7fc8dc06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4001bb0 tx=0x7fc8f4001110 comp rx=0 tx=0).stop 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 msgr2=0x7fc8f80c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c0e0 tx=0x7fc8e8001120 comp rx=0 tx=0).stop 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 shutdown_connections 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc8dc0682f0 0x7fc8dc06a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e4004850 0x7fc8f81e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8f8147340 0x7fc8f81477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 --2- 172.21.15.53:0/4120385142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8f80ca360 0x7fc8f80c8400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 >> 172.21.15.53:0/4120385142 conn(0x7fc8f8072600 msgr2=0x7fc8f8160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 shutdown_connections 2023-11-28T11:18:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.741+0000 7fc9007ff700 1 -- 172.21.15.53:0/4120385142 wait complete. 2023-11-28T11:18:35.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:35.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.968+0000 7fd8591ff700 1 Processor -- start 2023-11-28T11:18:35.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.968+0000 7fd8591ff700 1 -- start start 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.969+0000 7fd8591ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 0x7fd8540cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.969+0000 7fd8591ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540d1950 con 0x7fd854161340 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.969+0000 7fd8591ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540ce140 con 0x7fd85415c9b0 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.969+0000 7fd8591ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540b5800 con 0x7fd8540cd7c0 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd84bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd854161340 0x7fd8540d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43280/0 (socket says 172.21.15.53:43280) 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd84bfff700 1 -- 172.21.15.53:0/3618826495 learned_addr learned my addr 172.21.15.53:0/3618826495 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85359e700 1 --2- 172.21.15.53:0/3618826495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 0x7fd8540cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943846875 0 0) 0x7fd8540d1950 con 0x7fd854161340 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8400035b0 con 0x7fd854161340 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220977780 0 0) 0x7fd8540ce140 con 0x7fd85415c9b0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8540d1950 con 0x7fd85415c9b0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85359e700 1 -- 172.21.15.53:0/3618826495 >> v1:172.21.15.157:6789/0 conn(0x7fd85415c9b0 legacy=0x7fd8540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85359e700 1 -- 172.21.15.53:0/3618826495 >> v1:172.21.15.53:6789/0 conn(0x7fd854161340 legacy=0x7fd8540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.970+0000 7fd85359e700 1 -- 172.21.15.53:0/3618826495 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540d24f0 con 0x7fd8540cd7c0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd85359e700 1 --2- 172.21.15.53:0/3618826495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 0x7fd8540cdc00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009dc0 tx=0x7fd84c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c0de323d747766 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00e070 con 0x7fd8540cd7c0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00bb20 con 0x7fd8540cd7c0 2023-11-28T11:18:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd85259c700 1 -- 172.21.15.53:0/3618826495 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c012430 con 0x7fd8540cd7c0 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd8591ff700 1 -- 172.21.15.53:0/3618826495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 msgr2=0x7fd8540cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.971+0000 7fd8591ff700 1 --2- 172.21.15.53:0/3618826495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 0x7fd8540cdc00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009dc0 tx=0x7fd84c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.972+0000 7fd8591ff700 1 -- 172.21.15.53:0/3618826495 shutdown_connections 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.972+0000 7fd8591ff700 1 --2- 172.21.15.53:0/3618826495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8540cd7c0 0x7fd8540cdc00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.972+0000 7fd8591ff700 1 -- 172.21.15.53:0/3618826495 >> 172.21.15.53:0/3618826495 conn(0x7fd854072600 msgr2=0x7fd8540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.972+0000 7fd8591ff700 1 -- 172.21.15.53:0/3618826495 shutdown_connections 2023-11-28T11:18:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.972+0000 7fd8591ff700 1 -- 172.21.15.53:0/3618826495 wait complete. 2023-11-28T11:18:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.973+0000 7fd8591ff700 1 Processor -- start 2023-11-28T11:18:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.974+0000 7fd8591ff700 1 -- start start 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 0x7fd8540cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd854161340 0x7fd854157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541f2fe0 con 0x7fd8540cd7c0 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540ca240 con 0x7fd85415c9b0 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd8591ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540caa50 con 0x7fd854161340 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54086/0 (socket says 172.21.15.53:54086) 2023-11-28T11:18:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 -- 172.21.15.53:0/882055845 learned_addr learned my addr 172.21.15.53:0/882055845 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd85359e700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd854161340 0x7fd854157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd84bfff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 0x7fd8540cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 -- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd854161340 msgr2=0x7fd854157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd854161340 0x7fd854157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 -- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 msgr2=0x7fd8540cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 0x7fd8540cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd852d9d700 1 -- 172.21.15.53:0/882055845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c009a70 con 0x7fd8540cd7c0 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.975+0000 7fd84bfff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 0x7fd8540cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd852d9d700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c500 tx=0x7fd84400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440052d0 con 0x7fd8540cd7c0 2023-11-28T11:18:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844005450 con 0x7fd8540cd7c0 2023-11-28T11:18:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8540d1300 con 0x7fd8540cd7c0 2023-11-28T11:18:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440038c0 con 0x7fd8540cd7c0 2023-11-28T11:18:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.976+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8540b5520 con 0x7fd8540cd7c0 2023-11-28T11:18:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.977+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8540d15e0 con 0x7fd8540cd7c0 2023-11-28T11:18:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.978+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd844019030 con 0x7fd8540cd7c0 2023-11-28T11:18:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.979+0000 7fd84b7fe700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 0x7fd83406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.979+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd844084f60 con 0x7fd8540cd7c0 2023-11-28T11:18:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.979+0000 7fd84bfff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 0x7fd83406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:35.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.980+0000 7fd84bfff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 0x7fd83406a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd854159ff0 tx=0x7fd83c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:35.983+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd844051780 con 0x7fd8540cd7c0 2023-11-28T11:18:36.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.184+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8540745c0 con 0x7fd8540cd7c0 2023-11-28T11:18:36.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.184+0000 7fd84b7fe700 1 -- 172.21.15.53:0/882055845 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8440170a0 con 0x7fd8540cd7c0 2023-11-28T11:18:36.186 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 msgr2=0x7fd83406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 0x7fd83406a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd854159ff0 tx=0x7fd83c009040 comp rx=0 tx=0).stop 2023-11-28T11:18:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 msgr2=0x7fd8541f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c500 tx=0x7fd84400c810 comp rx=0 tx=0).stop 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 shutdown_connections 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd8340682f0 0x7fd83406a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd854161340 0x7fd854157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8540cd7c0 0x7fd8541f2aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 --2- 172.21.15.53:0/882055845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd85415c9b0 0x7fd8540cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 >> 172.21.15.53:0/882055845 conn(0x7fd854072600 msgr2=0x7fd8540d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:36.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 shutdown_connections 2023-11-28T11:18:36.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.187+0000 7fd8591ff700 1 -- 172.21.15.53:0/882055845 wait complete. 2023-11-28T11:18:36.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.409+0000 7fd5691ff700 1 Processor -- start 2023-11-28T11:18:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.410+0000 7fd5691ff700 1 -- start start 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd5691ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd564161340 0x7fd5640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd5691ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640d1950 con 0x7fd56415c9b0 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd5691ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640b5520 con 0x7fd5640d2320 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd5691ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5640b5800 con 0x7fd564161340 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd55bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd564161340 0x7fd5640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd562d9d700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd56415c9b0 0x7fd5640b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43282/0 (socket says 172.21.15.53:43282) 2023-11-28T11:18:36.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd562d9d700 1 -- 172.21.15.53:0/860939217 learned_addr learned my addr 172.21.15.53:0/860939217 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:36.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd55bfff700 1 -- 172.21.15.53:0/860939217 >> v1:172.21.15.157:6789/0 conn(0x7fd5640d2320 legacy=0x7fd5640cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd55bfff700 1 -- 172.21.15.53:0/860939217 >> v1:172.21.15.53:6789/0 conn(0x7fd56415c9b0 legacy=0x7fd5640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.411+0000 7fd55bfff700 1 -- 172.21.15.53:0/860939217 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640d1300 con 0x7fd564161340 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd55bfff700 1 --2- 172.21.15.53:0/860939217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd564161340 0x7fd5640d1de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009840 tx=0x7fd54c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21e258d89f46d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd56259c700 1 -- 172.21.15.53:0/860939217 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004230 con 0x7fd564161340 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd56259c700 1 -- 172.21.15.53:0/860939217 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c004390 con 0x7fd564161340 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd56259c700 1 -- 172.21.15.53:0/860939217 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0036a0 con 0x7fd564161340 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd5691ff700 1 -- 172.21.15.53:0/860939217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd564161340 msgr2=0x7fd5640d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd5691ff700 1 --2- 172.21.15.53:0/860939217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd564161340 0x7fd5640d1de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009840 tx=0x7fd54c009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd5691ff700 1 -- 172.21.15.53:0/860939217 shutdown_connections 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd5691ff700 1 --2- 172.21.15.53:0/860939217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd564161340 0x7fd5640d1de0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.412+0000 7fd5691ff700 1 -- 172.21.15.53:0/860939217 >> 172.21.15.53:0/860939217 conn(0x7fd564072600 msgr2=0x7fd5640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:36.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.413+0000 7fd5691ff700 1 -- 172.21.15.53:0/860939217 shutdown_connections 2023-11-28T11:18:36.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.413+0000 7fd5691ff700 1 -- 172.21.15.53:0/860939217 wait complete. 2023-11-28T11:18:36.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.414+0000 7fd5691ff700 1 Processor -- start 2023-11-28T11:18:36.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.415+0000 7fd5691ff700 1 -- start start 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.415+0000 7fd5691ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5640d2320 0x7fd5641e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd5691ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd5691ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd564161340 0x7fd5640c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd5691ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641e7e00 con 0x7fd56415c9b0 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd5691ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641e80e0 con 0x7fd564161340 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd5691ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641e83c0 con 0x7fd5640d2320 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54110/0 (socket says 172.21.15.53:54110) 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 -- 172.21.15.53:0/348724241 learned_addr learned my addr 172.21.15.53:0/348724241 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:36.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd562d9d700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5640d2320 0x7fd5641e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd56359e700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd564161340 0x7fd5640c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 -- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5640d2320 msgr2=0x7fd5641e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5640d2320 0x7fd5641e6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 -- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd564161340 msgr2=0x7fd5640c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd564161340 0x7fd5640c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 -- 172.21.15.53:0/348724241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0094a0 con 0x7fd56415c9b0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.416+0000 7fd55bfff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0090b0 tx=0x7fd54c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.417+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c011830 con 0x7fd56415c9b0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.417+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c01e070 con 0x7fd56415c9b0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.417+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56415cd80 con 0x7fd56415c9b0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.417+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c011bf0 con 0x7fd56415c9b0 2023-11-28T11:18:36.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.417+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5640d15e0 con 0x7fd56415c9b0 2023-11-28T11:18:36.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.418+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56415cf10 con 0x7fd56415c9b0 2023-11-28T11:18:36.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.419+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd54c017020 con 0x7fd56415c9b0 2023-11-28T11:18:36.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.420+0000 7fd55b7fe700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 0x7fd54406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.420+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd54c087160 con 0x7fd56415c9b0 2023-11-28T11:18:36.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.421+0000 7fd562d9d700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 0x7fd54406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.422+0000 7fd562d9d700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 0x7fd54406a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd554000f60 tx=0x7fd554009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.425+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd54c053890 con 0x7fd56415c9b0 2023-11-28T11:18:36.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.628+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5640745c0 con 0x7fd56415c9b0 2023-11-28T11:18:36.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.628+0000 7fd55b7fe700 1 -- 172.21.15.53:0/348724241 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd54c056eb0 con 0x7fd56415c9b0 2023-11-28T11:18:36.630 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:36.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 msgr2=0x7fd54406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 0x7fd54406a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd554000f60 tx=0x7fd554009040 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 msgr2=0x7fd5641e8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0090b0 tx=0x7fd54c004750 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 shutdown_connections 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd5440682f0 0x7fd54406a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5640d2320 0x7fd5641e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd564161340 0x7fd5640c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 --2- 172.21.15.53:0/348724241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd56415c9b0 0x7fd5641e8ea0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 >> 172.21.15.53:0/348724241 conn(0x7fd564072600 msgr2=0x7fd5640b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.630+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 shutdown_connections 2023-11-28T11:18:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.631+0000 7fd5691ff700 1 -- 172.21.15.53:0/348724241 wait complete. 2023-11-28T11:18:36.650 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:36.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.855+0000 7f35aa3ff700 1 Processor -- start 2023-11-28T11:18:36.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.855+0000 7f35aa3ff700 1 -- start start 2023-11-28T11:18:36.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.856+0000 7f35aa3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a415c9b0 0x7f35a40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.856+0000 7f35aa3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35a4161340 0x7f35a40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.856+0000 7f35aa3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40d1950 con 0x7f35a40d2320 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.856+0000 7f35aa3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40b5800 con 0x7f35a4161340 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.856+0000 7f35aa3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b5520 con 0x7f35a415c9b0 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35a4161340 0x7f35a40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a899c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f35a40d2320 0x7f35a40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43298/0 (socket says 172.21.15.53:43298) 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a899c700 1 -- 172.21.15.53:0/2676941327 learned_addr learned my addr 172.21.15.53:0/2676941327 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a3fff700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a415c9b0 0x7f35a40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 -- 172.21.15.53:0/2676941327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a415c9b0 msgr2=0x7f35a40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a415c9b0 0x7f35a40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 -- 172.21.15.53:0/2676941327 >> v1:172.21.15.53:6789/0 conn(0x7f35a40d2320 legacy=0x7f35a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 -- 172.21.15.53:0/2676941327 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40d1300 con 0x7f35a4161340 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a2ffd700 1 -- 172.21.15.53:0/2676941327 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062291464 0 0) 0x7f35a40d1950 con 0x7f35a40d2320 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.857+0000 7f35a37fe700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40d1de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3598009840 tx=0x7f3598009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aa49257caaaf8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35a2ffd700 1 -- 172.21.15.53:0/2676941327 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011830 con 0x7f35a4161340 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35a2ffd700 1 -- 172.21.15.53:0/2676941327 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598011990 con 0x7f35a4161340 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35a2ffd700 1 -- 172.21.15.53:0/2676941327 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011d20 con 0x7f35a4161340 2023-11-28T11:18:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 -- 172.21.15.53:0/2676941327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 msgr2=0x7f35a40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40d1de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3598009840 tx=0x7f3598009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 -- 172.21.15.53:0/2676941327 shutdown_connections 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a415c9b0 0x7f35a40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/2676941327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40d1de0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 -- 172.21.15.53:0/2676941327 >> 172.21.15.53:0/2676941327 conn(0x7f35a4072600 msgr2=0x7f35a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 -- 172.21.15.53:0/2676941327 shutdown_connections 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.858+0000 7f35aa3ff700 1 -- 172.21.15.53:0/2676941327 wait complete. 2023-11-28T11:18:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.860+0000 7f35aa3ff700 1 Processor -- start 2023-11-28T11:18:36.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.860+0000 7f35aa3ff700 1 -- start start 2023-11-28T11:18:36.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a40d2320 0x7f35a4149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35a415c9b0 0x7f35a41567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a414a440 con 0x7f35a4161340 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a41f23a0 con 0x7f35a415c9b0 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.861+0000 7f35aa3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4156ff0 con 0x7f35a40d2320 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54130/0 (socket says 172.21.15.53:54130) 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 -- 172.21.15.53:0/754631917 learned_addr learned my addr 172.21.15.53:0/754631917 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a3fff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a40d2320 0x7f35a4149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a37fe700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35a415c9b0 0x7f35a41567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 -- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a40d2320 msgr2=0x7f35a4149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a40d2320 0x7f35a4149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 -- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35a415c9b0 msgr2=0x7f35a41567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35a415c9b0 0x7f35a41567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 -- 172.21.15.53:0/754631917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980094a0 con 0x7f35a4161340 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a899c700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f359c00cb60 tx=0x7f359c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.862+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00a9e0 con 0x7f35a4161340 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.863+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00ab40 con 0x7f35a4161340 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.863+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a40b5130 con 0x7f35a4161340 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.863+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0070d0 con 0x7f35a4161340 2023-11-28T11:18:36.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.863+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a40d15e0 con 0x7f35a4161340 2023-11-28T11:18:36.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.864+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a40b52c0 con 0x7f35a4161340 2023-11-28T11:18:36.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.865+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f359c01a030 con 0x7f35a4161340 2023-11-28T11:18:36.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.865+0000 7f35a0ff9700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:36.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.866+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f359c085910 con 0x7f35a4161340 2023-11-28T11:18:36.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.866+0000 7f35a3fff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:36.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.867+0000 7f35a3fff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 0x7f359006a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3594006b20 tx=0x7f3594008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:36.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:36.869+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f359c052040 con 0x7f35a4161340 2023-11-28T11:18:37.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.080+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35a40745c0 con 0x7f35a4161340 2023-11-28T11:18:37.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.081+0000 7f35a0ff9700 1 -- 172.21.15.53:0/754631917 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f359c0180a0 con 0x7f35a4161340 2023-11-28T11:18:37.083 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:37.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 msgr2=0x7f359006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 0x7f359006a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3594006b20 tx=0x7f3594008040 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 msgr2=0x7f35a40b8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f359c00cb60 tx=0x7f359c00ce70 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 shutdown_connections 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35a40d2320 0x7f35a4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a4161340 0x7f35a40b8170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 --2- 172.21.15.53:0/754631917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35a415c9b0 0x7f35a41567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 >> 172.21.15.53:0/754631917 conn(0x7f35a4072600 msgr2=0x7f35a415fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 shutdown_connections 2023-11-28T11:18:37.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.083+0000 7f35aa3ff700 1 -- 172.21.15.53:0/754631917 wait complete. 2023-11-28T11:18:37.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.308+0000 7f19eedff700 1 Processor -- start 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.308+0000 7f19eedff700 1 -- start start 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.309+0000 7f19eedff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19e815c9b0 0x7f19e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.309+0000 7f19eedff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 0x7f19e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.309+0000 7f19eedff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e80d1950 con 0x7f19e80d2320 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.309+0000 7f19eedff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80b5800 con 0x7f19e815c9b0 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.309+0000 7f19eedff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80b5520 con 0x7f19e8161340 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19e815c9b0 0x7f19e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19e815c9b0 0x7f19e80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54142/0 (socket says 172.21.15.53:54142) 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 -- 172.21.15.53:0/1191232688 learned_addr learned my addr 172.21.15.53:0/1191232688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 -- 172.21.15.53:0/1191232688 >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 msgr2=0x7f19e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1191232688 >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 0x7f19e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1191232688 >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 0x7f19e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 -- 172.21.15.53:0/1191232688 >> v1:172.21.15.157:6789/0 conn(0x7f19e80d2320 legacy=0x7f19e80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19ecb9b700 1 -- 172.21.15.53:0/1191232688 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80d1300 con 0x7f19e815c9b0 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.310+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1191232688 >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 0x7f19e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1191232688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e815c9b0 0x7f19e80b4bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f19e80685c0 tx=0x7f19dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f36fe1c49eebd78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19e77fe700 1 -- 172.21.15.53:0/1191232688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc004230 con 0x7f19e815c9b0 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19e77fe700 1 -- 172.21.15.53:0/1191232688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc004390 con 0x7f19e815c9b0 2023-11-28T11:18:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19e77fe700 1 -- 172.21.15.53:0/1191232688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0036a0 con 0x7f19e815c9b0 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 -- 172.21.15.53:0/1191232688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e815c9b0 msgr2=0x7f19e80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 --2- 172.21.15.53:0/1191232688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e815c9b0 0x7f19e80b4bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f19e80685c0 tx=0x7f19dc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 -- 172.21.15.53:0/1191232688 shutdown_connections 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 --2- 172.21.15.53:0/1191232688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e815c9b0 0x7f19e80b4bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 --2- 172.21.15.53:0/1191232688 >> v2:172.21.15.157:3300/0 conn(0x7f19e8161340 0x7f19e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.311+0000 7f19eedff700 1 -- 172.21.15.53:0/1191232688 >> 172.21.15.53:0/1191232688 conn(0x7f19e8072600 msgr2=0x7f19e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.312+0000 7f19eedff700 1 -- 172.21.15.53:0/1191232688 shutdown_connections 2023-11-28T11:18:37.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.312+0000 7f19eedff700 1 -- 172.21.15.53:0/1191232688 wait complete. 2023-11-28T11:18:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.313+0000 7f19eedff700 1 Processor -- start 2023-11-28T11:18:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.314+0000 7f19eedff700 1 -- start start 2023-11-28T11:18:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.314+0000 7f19eedff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 0x7f19e80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.314+0000 7f19eedff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 0x7f19e8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19eedff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19eedff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8150860 con 0x7f19e8161340 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19eedff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8150d60 con 0x7f19e815c9b0 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19eedff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81547c0 con 0x7f19e80d2320 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54158/0 (socket says 172.21.15.53:54158) 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 -- 172.21.15.53:0/1484803822 learned_addr learned my addr 172.21.15.53:0/1484803822 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 0x7f19e8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 0x7f19e80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 -- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 msgr2=0x7f19e80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 0x7f19e80c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 -- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 msgr2=0x7f19e8154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 0x7f19e8154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ed39c700 1 -- 172.21.15.53:0/1484803822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009550 con 0x7f19e8161340 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 0x7f19e80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.315+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 0x7f19e8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19ed39c700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b9e0 tx=0x7f19e000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000c9b0 con 0x7f19e8161340 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e000cb50 con 0x7f19e8161340 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80b5130 con 0x7f19e8161340 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0014440 con 0x7f19e8161340 2023-11-28T11:18:37.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.316+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e80d15e0 con 0x7f19e8161340 2023-11-28T11:18:37.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.317+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e80b52c0 con 0x7f19e8161340 2023-11-28T11:18:37.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.318+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19e001b030 con 0x7f19e8161340 2023-11-28T11:18:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.320+0000 7f19e57fa700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.320+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f19e00858a0 con 0x7f19e8161340 2023-11-28T11:18:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.320+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.321+0000 7f19ecb9b700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 0x7f19cc06a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009c10 tx=0x7f19dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:37.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.324+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19e004d5e0 con 0x7f19e8161340 2023-11-28T11:18:37.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.543+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f19e80745c0 con 0x7f19e8161340 2023-11-28T11:18:37.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.544+0000 7f19e57fa700 1 -- 172.21.15.53:0/1484803822 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19e0052080 con 0x7f19e8161340 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 msgr2=0x7f19cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 0x7f19cc06a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009c10 tx=0x7f19dc008030 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 msgr2=0x7f19e8150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b9e0 tx=0x7f19e000bda0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 shutdown_connections 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80d2320 0x7f19e80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e8161340 0x7f19e8150320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 --2- 172.21.15.53:0/1484803822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e815c9b0 0x7f19e8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:37.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 >> 172.21.15.53:0/1484803822 conn(0x7f19e8072600 msgr2=0x7f19e80b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:37.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 shutdown_connections 2023-11-28T11:18:37.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:37.546+0000 7f19eedff700 1 -- 172.21.15.53:0/1484803822 wait complete. 2023-11-28T11:18:37.549 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:18:37.565 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T11:18:37.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:37.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T11:18:37.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:18:37.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:37.638 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:18:37.682 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:37.683 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:37.732 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:37.733 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:37.778 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:18:37.778 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:18:37.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:38.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.063+0000 7fac2d7ff700 1 Processor -- start 2023-11-28T11:18:38.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.064+0000 7fac2d7ff700 1 -- start start 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac2d7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 0x7fac280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac2d7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280d1950 con 0x7fac2815c9b0 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac2d7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280b5520 con 0x7fac280d2320 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac2d7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280b5800 con 0x7fac28161340 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac26ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fac2815c9b0 0x7fac280b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43306/0 (socket says 172.21.15.53:43306) 2023-11-28T11:18:38.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac26ffd700 1 -- 172.21.15.53:0/1631271364 learned_addr learned my addr 172.21.15.53:0/1631271364 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac1ffff700 1 --2- 172.21.15.53:0/1631271364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 0x7fac280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210003688 0 0) 0x7fac280d1950 con 0x7fac2815c9b0 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.065+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac140035b0 con 0x7fac2815c9b0 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087645668 0 0) 0x7fac280b5520 con 0x7fac280d2320 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac280d1950 con 0x7fac280d2320 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac1ffff700 1 -- 172.21.15.53:0/1631271364 >> v1:172.21.15.157:6789/0 conn(0x7fac280d2320 legacy=0x7fac280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228969678 0 0) 0x7fac140035b0 con 0x7fac2815c9b0 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac1ffff700 1 -- 172.21.15.53:0/1631271364 >> v1:172.21.15.53:6789/0 conn(0x7fac2815c9b0 legacy=0x7fac280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac1ffff700 1 -- 172.21.15.53:0/1631271364 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280d1300 con 0x7fac28161340 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac1ffff700 1 --2- 172.21.15.53:0/1631271364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 0x7fac280d1de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac10009870 tx=0x7fac10009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5021a1ae61179b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18001670 con 0x7fac28161340 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10004030 con 0x7fac28161340 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.066+0000 7fac267fc700 1 -- 172.21.15.53:0/1631271364 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100043c0 con 0x7fac28161340 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1631271364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 msgr2=0x7fac280d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1631271364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 0x7fac280d1de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac10009870 tx=0x7fac10009b80 comp rx=0 tx=0).stop 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1631271364 shutdown_connections 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1631271364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac28161340 0x7fac280d1de0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1631271364 >> 172.21.15.53:0/1631271364 conn(0x7fac28072600 msgr2=0x7fac280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1631271364 shutdown_connections 2023-11-28T11:18:38.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.067+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1631271364 wait complete. 2023-11-28T11:18:38.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.069+0000 7fac2d7ff700 1 Processor -- start 2023-11-28T11:18:38.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.069+0000 7fac2d7ff700 1 -- start start 2023-11-28T11:18:38.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 0x7fac28156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2815c9b0 0x7fac280c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280c7d40 con 0x7fac28161340 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281570d0 con 0x7fac2815c9b0 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac2d7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280c6ca0 con 0x7fac280d2320 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac277fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac277fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54168/0 (socket says 172.21.15.53:54168) 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac277fe700 1 -- 172.21.15.53:0/1871011652 learned_addr learned my addr 172.21.15.53:0/1871011652 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac1ffff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2815c9b0 0x7fac280c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.070+0000 7fac26ffd700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 0x7fac28156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 -- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 msgr2=0x7fac28156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 0x7fac28156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 -- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2815c9b0 msgr2=0x7fac280c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2815c9b0 0x7fac280c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 -- 172.21.15.53:0/1871011652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100094a0 con 0x7fac28161340 2023-11-28T11:18:38.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac26ffd700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 0x7fac28156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:38.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac277fe700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac2000cbc0 tx=0x7fac2000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20003c00 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20003d60 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000ab00 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2815cd80 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.071+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac280d15e0 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.072+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2815cf10 con 0x7fac28161340 2023-11-28T11:18:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.073+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac20010050 con 0x7fac28161340 2023-11-28T11:18:38.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.074+0000 7fac1f7fe700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 0x7fac0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.074+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fac20085ce0 con 0x7fac28161340 2023-11-28T11:18:38.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.075+0000 7fac26ffd700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 0x7fac0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.075+0000 7fac26ffd700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 0x7fac0806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac18001860 tx=0x7fac180016d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.078+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac20052410 con 0x7fac28161340 2023-11-28T11:18:38.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.284+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac280745c0 con 0x7fac28161340 2023-11-28T11:18:38.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.286+0000 7fac1f7fe700 1 -- 172.21.15.53:0/1871011652 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18002 (secure 0 0 0) 0x7fac20055a30 con 0x7fac28161340 2023-11-28T11:18:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:38.287 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:36.870456+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.9","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 msgr2=0x7fac0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 0x7fac0806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac18001860 tx=0x7fac180016d0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 msgr2=0x7fac2815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac2000cbc0 tx=0x7fac2000ced0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 shutdown_connections 2023-11-28T11:18:38.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac080682f0 0x7fac0806a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280d2320 0x7fac28156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac28161340 0x7fac2815f0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 --2- 172.21.15.53:0/1871011652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2815c9b0 0x7fac280c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 >> 172.21.15.53:0/1871011652 conn(0x7fac28072600 msgr2=0x7fac2806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 shutdown_connections 2023-11-28T11:18:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.288+0000 7fac2d7ff700 1 -- 172.21.15.53:0/1871011652 wait complete. 2023-11-28T11:18:38.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:38.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.512+0000 7fefbddff700 1 Processor -- start 2023-11-28T11:18:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.512+0000 7fefbddff700 1 -- start start 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.513+0000 7fefbddff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefb815c9b0 0x7fefb80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.513+0000 7fefbddff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fefb8161340 0x7fefb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefbddff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb80d1950 con 0x7fefb80d2320 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefbddff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb80b5800 con 0x7fefb8161340 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefbddff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb80b5520 con 0x7fefb815c9b0 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefaffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fefb8161340 0x7fefb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefb7fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fefb80d2320 0x7fefb80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43310/0 (socket says 172.21.15.53:43310) 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefb7fff700 1 -- 172.21.15.53:0/1105272220 learned_addr learned my addr 172.21.15.53:0/1105272220 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefb77fe700 1 --2- 172.21.15.53:0/1105272220 >> v2:172.21.15.157:3300/0 conn(0x7fefb815c9b0 0x7fefb80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefaffff700 1 -- 172.21.15.53:0/1105272220 >> v2:172.21.15.157:3300/0 conn(0x7fefb815c9b0 msgr2=0x7fefb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefaffff700 1 --2- 172.21.15.53:0/1105272220 >> v2:172.21.15.157:3300/0 conn(0x7fefb815c9b0 0x7fefb80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefaffff700 1 -- 172.21.15.53:0/1105272220 >> v1:172.21.15.53:6789/0 conn(0x7fefb80d2320 legacy=0x7fefb80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefb6ffd700 1 -- 172.21.15.53:0/1105272220 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845849491 0 0) 0x7fefb80d1950 con 0x7fefb80d2320 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.514+0000 7fefaffff700 1 -- 172.21.15.53:0/1105272220 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80d1300 con 0x7fefb8161340 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefaffff700 1 --2- 172.21.15.53:0/1105272220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb8161340 0x7fefb80d1de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefb8067460 tx=0x7fefa0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b3668dbb57b1a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefb6ffd700 1 -- 172.21.15.53:0/1105272220 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011830 con 0x7fefb8161340 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefb6ffd700 1 -- 172.21.15.53:0/1105272220 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0011990 con 0x7fefb8161340 2023-11-28T11:18:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefb6ffd700 1 -- 172.21.15.53:0/1105272220 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011d20 con 0x7fefb8161340 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefbddff700 1 -- 172.21.15.53:0/1105272220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb8161340 msgr2=0x7fefb80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.515+0000 7fefbddff700 1 --2- 172.21.15.53:0/1105272220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb8161340 0x7fefb80d1de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefb8067460 tx=0x7fefa0009b20 comp rx=0 tx=0).stop 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 -- 172.21.15.53:0/1105272220 shutdown_connections 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 --2- 172.21.15.53:0/1105272220 >> v2:172.21.15.157:3300/0 conn(0x7fefb815c9b0 0x7fefb80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 --2- 172.21.15.53:0/1105272220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb8161340 0x7fefb80d1de0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 -- 172.21.15.53:0/1105272220 >> 172.21.15.53:0/1105272220 conn(0x7fefb8072600 msgr2=0x7fefb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 -- 172.21.15.53:0/1105272220 shutdown_connections 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.516+0000 7fefbddff700 1 -- 172.21.15.53:0/1105272220 wait complete. 2023-11-28T11:18:38.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.517+0000 7fefbddff700 1 Processor -- start 2023-11-28T11:18:38.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.517+0000 7fefbddff700 1 -- start start 2023-11-28T11:18:38.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb80d2320 0x7fefb80c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8161340 0x7fefb80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8150bb0 con 0x7fefb815c9b0 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80c7d60 con 0x7fefb8161340 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.518+0000 7fefbddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80c9240 con 0x7fefb80d2320 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54178/0 (socket says 172.21.15.53:54178) 2023-11-28T11:18:38.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 -- 172.21.15.53:0/528939762 learned_addr learned my addr 172.21.15.53:0/528939762 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 -- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb80d2320 msgr2=0x7fefb80c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb80d2320 0x7fefb80c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 -- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8161340 msgr2=0x7fefb80c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8161340 0x7fefb80c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 -- 172.21.15.53:0/528939762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00094a0 con 0x7fefb815c9b0 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.519+0000 7fefaffff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00090b0 tx=0x7fefa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.520+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00038a0 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.520+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa001f070 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.520+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003c20 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.520+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb80b5130 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.520+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb80d15e0 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.521+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb80b52c0 con 0x7fefb815c9b0 2023-11-28T11:18:38.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.522+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fefa000f050 con 0x7fefb815c9b0 2023-11-28T11:18:38.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.522+0000 7fefb4ff9700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.522+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fefa0021070 con 0x7fefb815c9b0 2023-11-28T11:18:38.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.523+0000 7fefb77fe700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.524+0000 7fefb77fe700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 0x7fef9806a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006b20 tx=0x7fefa8006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.526+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fefa0053e90 con 0x7fefb815c9b0 2023-11-28T11:18:38.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.729+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefb80745c0 con 0x7fefb815c9b0 2023-11-28T11:18:38.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.729+0000 7fefb4ff9700 1 -- 172.21.15.53:0/528939762 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefa00574b0 con 0x7fefb815c9b0 2023-11-28T11:18:38.731 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 msgr2=0x7fef9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 0x7fef9806a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006b20 tx=0x7fefa8006ab0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 msgr2=0x7fefb80b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00090b0 tx=0x7fefa0004750 comp rx=0 tx=0).stop 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 shutdown_connections 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefb80d2320 0x7fefb80c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb8161340 0x7fefb80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 --2- 172.21.15.53:0/528939762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb815c9b0 0x7fefb80b76a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 >> 172.21.15.53:0/528939762 conn(0x7fefb8072600 msgr2=0x7fefb815a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:38.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 shutdown_connections 2023-11-28T11:18:38.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.732+0000 7fefbddff700 1 -- 172.21.15.53:0/528939762 wait complete. 2023-11-28T11:18:38.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:38.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.955+0000 7fd3039ff700 1 Processor -- start 2023-11-28T11:18:38.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.955+0000 7fd3039ff700 1 -- start start 2023-11-28T11:18:38.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.956+0000 7fd3039ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.956+0000 7fd3039ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2fc161340 0x7fd2fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.956+0000 7fd3039ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0d1950 con 0x7fd2fc0d2320 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.956+0000 7fd3039ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0b5800 con 0x7fd2fc161340 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.956+0000 7fd3039ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0b5520 con 0x7fd2fc15c9b0 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2fc161340 0x7fd2fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd301f9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd2fc0d2320 0x7fd2fc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43312/0 (socket says 172.21.15.53:43312) 2023-11-28T11:18:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd301f9c700 1 -- 172.21.15.53:0/2409596431 learned_addr learned my addr 172.21.15.53:0/2409596431 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd30179b700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 -- 172.21.15.53:0/2409596431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 msgr2=0x7fd2fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 -- 172.21.15.53:0/2409596431 >> v1:172.21.15.53:6789/0 conn(0x7fd2fc0d2320 legacy=0x7fd2fc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 -- 172.21.15.53:0/2409596431 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0d1300 con 0x7fd2fc161340 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.957+0000 7fd300f9a700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc0d1de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009840 tx=0x7fd2ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6bef7490bde1c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd2f3fff700 1 -- 172.21.15.53:0/2409596431 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec011830 con 0x7fd2fc161340 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd2f3fff700 1 -- 172.21.15.53:0/2409596431 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec011990 con 0x7fd2fc161340 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd2f3fff700 1 -- 172.21.15.53:0/2409596431 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec011d20 con 0x7fd2fc161340 2023-11-28T11:18:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 -- 172.21.15.53:0/2409596431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 msgr2=0x7fd2fc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc0d1de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009840 tx=0x7fd2ec009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 -- 172.21.15.53:0/2409596431 shutdown_connections 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 --2- 172.21.15.53:0/2409596431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc0d1de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 -- 172.21.15.53:0/2409596431 >> 172.21.15.53:0/2409596431 conn(0x7fd2fc072600 msgr2=0x7fd2fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 -- 172.21.15.53:0/2409596431 shutdown_connections 2023-11-28T11:18:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.958+0000 7fd3039ff700 1 -- 172.21.15.53:0/2409596431 wait complete. 2023-11-28T11:18:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.960+0000 7fd3039ff700 1 Processor -- start 2023-11-28T11:18:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.960+0000 7fd3039ff700 1 -- start start 2023-11-28T11:18:38.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0d2320 0x7fd2fc0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1584b0 con 0x7fd2fc161340 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c9ad0 con 0x7fd2fc0d2320 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.961+0000 7fd3039ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c8bc0 con 0x7fd2fc15c9b0 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54184/0 (socket says 172.21.15.53:54184) 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 -- 172.21.15.53:0/1343227842 learned_addr learned my addr 172.21.15.53:0/1343227842 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd30179b700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0d2320 0x7fd2fc0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd300f9a700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 -- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 msgr2=0x7fd2fc0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 -- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0d2320 msgr2=0x7fd2fc0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0d2320 0x7fd2fc0c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 -- 172.21.15.53:0/1343227842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0094a0 con 0x7fd2fc161340 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.962+0000 7fd301f9c700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ca80 tx=0x7fd2f400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.963+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400a9e0 con 0x7fd2fc161340 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.963+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f400ab40 con 0x7fd2fc161340 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.963+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc0b5130 con 0x7fd2fc161340 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.963+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4012400 con 0x7fd2fc161340 2023-11-28T11:18:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.963+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc0d15e0 con 0x7fd2fc161340 2023-11-28T11:18:38.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.964+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc0b52c0 con 0x7fd2fc161340 2023-11-28T11:18:38.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.965+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2f4019030 con 0x7fd2fc161340 2023-11-28T11:18:38.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.965+0000 7fd2f1ffb700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 0x7fd2e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:38.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.966+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd2f40856b0 con 0x7fd2fc161340 2023-11-28T11:18:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.966+0000 7fd30179b700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 0x7fd2e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.967+0000 7fd30179b700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 0x7fd2e006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8006b20 tx=0x7fd2f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:38.969+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2f404b5e0 con 0x7fd2fc161340 2023-11-28T11:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.170+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2fc0745c0 con 0x7fd2fc161340 2023-11-28T11:18:39.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.171+0000 7fd2f1ffb700 1 -- 172.21.15.53:0/1343227842 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2f4051e90 con 0x7fd2fc161340 2023-11-28T11:18:39.172 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.173+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 msgr2=0x7fd2e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.173+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 0x7fd2e006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8006b20 tx=0x7fd2f8008040 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.173+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 msgr2=0x7fd2fc150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.173+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ca80 tx=0x7fd2f400cd90 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 shutdown_connections 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2e00682f0 0x7fd2e006a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2fc161340 0x7fd2fc150f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0d2320 0x7fd2fc0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 --2- 172.21.15.53:0/1343227842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2fc15c9b0 0x7fd2fc0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 >> 172.21.15.53:0/1343227842 conn(0x7fd2fc072600 msgr2=0x7fd2fc15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 shutdown_connections 2023-11-28T11:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.174+0000 7fd3039ff700 1 -- 172.21.15.53:0/1343227842 wait complete. 2023-11-28T11:18:39.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:39.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.396+0000 7fc7ccdff700 1 Processor -- start 2023-11-28T11:18:39.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.396+0000 7fc7ccdff700 1 -- start start 2023-11-28T11:18:39.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.397+0000 7fc7ccdff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c80b20b0 0x7fc7c80b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.397+0000 7fc7ccdff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b0230 con 0x7fc7c80d1330 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.397+0000 7fc7ccdff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b4570 con 0x7fc7c80d01b0 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.397+0000 7fc7ccdff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c815cec0 con 0x7fc7c80b20b0 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c5d9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc7c80d1330 0x7fc7c80b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43328/0 (socket says 172.21.15.53:43328) 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c5d9b700 1 -- 172.21.15.53:0/430776 learned_addr learned my addr 172.21.15.53:0/430776 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c6d9d700 1 --2- 172.21.15.53:0/430776 >> v2:172.21.15.157:3300/0 conn(0x7fc7c80b20b0 0x7fc7c80b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698358874 0 0) 0x7fc7c80b0230 con 0x7fc7c80d1330 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b00035b0 con 0x7fc7c80d1330 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584283749 0 0) 0x7fc7c80b4570 con 0x7fc7c80d01b0 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c80b0230 con 0x7fc7c80d01b0 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/430776 >> v1:172.21.15.157:6789/0 conn(0x7fc7c80d01b0 legacy=0x7fc7c80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/430776 >> v1:172.21.15.53:6789/0 conn(0x7fc7c80d1330 legacy=0x7fc7c80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.398+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/430776 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80b4850 con 0x7fc7c80b20b0 2023-11-28T11:18:39.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7c6d9d700 1 --2- 172.21.15.53:0/430776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80b20b0 0x7fc7c80b0dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009d90 tx=0x7fc7c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c71c67b7177819 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000e070 con 0x7fc7c80b20b0 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000bb20 con 0x7fc7c80b20b0 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7c559a700 1 -- 172.21.15.53:0/430776 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0012430 con 0x7fc7c80b20b0 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 -- 172.21.15.53:0/430776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80b20b0 msgr2=0x7fc7c80b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/430776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80b20b0 0x7fc7c80b0dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009d90 tx=0x7fc7c000b330 comp rx=0 tx=0).stop 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 -- 172.21.15.53:0/430776 shutdown_connections 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/430776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80b20b0 0x7fc7c80b0dd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 -- 172.21.15.53:0/430776 >> 172.21.15.53:0/430776 conn(0x7fc7c809c3f0 msgr2=0x7fc7c80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.399+0000 7fc7ccdff700 1 -- 172.21.15.53:0/430776 shutdown_connections 2023-11-28T11:18:39.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.400+0000 7fc7ccdff700 1 -- 172.21.15.53:0/430776 wait complete. 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.401+0000 7fc7ccdff700 1 Processor -- start 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.401+0000 7fc7ccdff700 1 -- start start 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80b20b0 0x7fc7c814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d01b0 0x7fc7c80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c814eef0 con 0x7fc7c80d1330 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b83a0 con 0x7fc7c80d01b0 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.402+0000 7fc7ccdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b8680 con 0x7fc7c80b20b0 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54190/0 (socket says 172.21.15.53:54190) 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/2420349887 learned_addr learned my addr 172.21.15.53:0/2420349887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c659c700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80b20b0 0x7fc7c814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c5d9b700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d01b0 0x7fc7c80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80b20b0 msgr2=0x7fc7c814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80b20b0 0x7fc7c814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d01b0 msgr2=0x7fc7c80b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d01b0 0x7fc7c80b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 -- 172.21.15.53:0/2420349887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009a70 con 0x7fc7c80d1330 2023-11-28T11:18:39.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.403+0000 7fc7c6d9d700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0013000 tx=0x7fc7c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.404+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c001b810 con 0x7fc7c80d1330 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.404+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0020070 con 0x7fc7c80d1330 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.404+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80afec0 con 0x7fc7c80d1330 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.404+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c001bbb0 con 0x7fc7c80d1330 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.404+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c80afbe0 con 0x7fc7c80d1330 2023-11-28T11:18:39.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.405+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c80b2600 con 0x7fc7c80d1330 2023-11-28T11:18:39.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.406+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc7c002e070 con 0x7fc7c80d1330 2023-11-28T11:18:39.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.408+0000 7fc7b6ffd700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 0x7fc7a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.408+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc7c0022070 con 0x7fc7c80d1330 2023-11-28T11:18:39.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.408+0000 7fc7c659c700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 0x7fc7a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.409+0000 7fc7c659c700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 0x7fc7a806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80011e0 tx=0x7fc7b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.411+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7c00549a0 con 0x7fc7c80d1330 2023-11-28T11:18:39.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.615+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7c8068360 con 0x7fc7c80d1330 2023-11-28T11:18:39.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.615+0000 7fc7b6ffd700 1 -- 172.21.15.53:0/2420349887 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7c0057fc0 con 0x7fc7c80d1330 2023-11-28T11:18:39.617 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:39.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 msgr2=0x7fc7a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 0x7fc7a806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80011e0 tx=0x7fc7b8009040 comp rx=0 tx=0).stop 2023-11-28T11:18:39.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 msgr2=0x7fc7c809f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0013000 tx=0x7fc7c0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 shutdown_connections 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc7a80682f0 0x7fc7a806a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80b20b0 0x7fc7c814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c80d1330 0x7fc7c809f760 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 --2- 172.21.15.53:0/2420349887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d01b0 0x7fc7c80b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 >> 172.21.15.53:0/2420349887 conn(0x7fc7c809c3f0 msgr2=0x7fc7c80b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:39.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 shutdown_connections 2023-11-28T11:18:39.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.618+0000 7fc7ccdff700 1 -- 172.21.15.53:0/2420349887 wait complete. 2023-11-28T11:18:39.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.841+0000 7f5be4fff700 1 Processor -- start 2023-11-28T11:18:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.841+0000 7f5be4fff700 1 -- start start 2023-11-28T11:18:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5be4fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5be00cd7c0 0x7f5be00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5be4fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00d1950 con 0x7f5be015c9b0 2023-11-28T11:18:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5be4fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00ce140 con 0x7f5be0161340 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5be4fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be00b5800 con 0x7f5be00cd7c0 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5bde59c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5be015c9b0 0x7f5be00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43332/0 (socket says 172.21.15.53:43332) 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.842+0000 7f5bde59c700 1 -- 172.21.15.53:0/286386154 learned_addr learned my addr 172.21.15.53:0/286386154 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bded9d700 1 --2- 172.21.15.53:0/286386154 >> v2:172.21.15.157:3300/0 conn(0x7f5be00cd7c0 0x7f5be00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766411723 0 0) 0x7f5be00d1950 con 0x7f5be015c9b0 2023-11-28T11:18:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bcc0035b0 con 0x7f5be015c9b0 2023-11-28T11:18:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bded9d700 1 -- 172.21.15.53:0/286386154 >> v1:172.21.15.157:6789/0 conn(0x7f5be0161340 legacy=0x7f5be00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193846193 0 0) 0x7f5bcc0035b0 con 0x7f5be015c9b0 2023-11-28T11:18:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bded9d700 1 -- 172.21.15.53:0/286386154 >> v1:172.21.15.53:6789/0 conn(0x7f5be015c9b0 legacy=0x7f5be00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bded9d700 1 -- 172.21.15.53:0/286386154 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00d24f0 con 0x7f5be00cd7c0 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.843+0000 7f5bded9d700 1 --2- 172.21.15.53:0/286386154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be00cd7c0 0x7f5be00cdc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009d90 tx=0x7f5bd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4abd1918618227 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800e070 con 0x7f5be00cd7c0 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8003a20 con 0x7f5be00cd7c0 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5bddd9b700 1 -- 172.21.15.53:0/286386154 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0001670 con 0x7f5be00cd7c0 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 -- 172.21.15.53:0/286386154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be00cd7c0 msgr2=0x7f5be00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 --2- 172.21.15.53:0/286386154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be00cd7c0 0x7f5be00cdc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009d90 tx=0x7f5bd800b330 comp rx=0 tx=0).stop 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 -- 172.21.15.53:0/286386154 shutdown_connections 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 --2- 172.21.15.53:0/286386154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be00cd7c0 0x7f5be00cdc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 -- 172.21.15.53:0/286386154 >> 172.21.15.53:0/286386154 conn(0x7f5be0072600 msgr2=0x7f5be00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 -- 172.21.15.53:0/286386154 shutdown_connections 2023-11-28T11:18:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.844+0000 7f5be4fff700 1 -- 172.21.15.53:0/286386154 wait complete. 2023-11-28T11:18:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.846+0000 7f5be4fff700 1 Processor -- start 2023-11-28T11:18:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.846+0000 7f5be4fff700 1 -- start start 2023-11-28T11:18:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 0x7f5be0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0161340 0x7f5be01f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0157c90 con 0x7f5be015c9b0 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00ca060 con 0x7f5be0161340 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5be4fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01f3080 con 0x7f5be00cd7c0 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.847+0000 7f5bd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54196/0 (socket says 172.21.15.53:54196) 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 -- 172.21.15.53:0/783656521 learned_addr learned my addr 172.21.15.53:0/783656521 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bded9d700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0161340 0x7f5be01f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bde59c700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 0x7f5be0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 -- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 msgr2=0x7f5be0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 0x7f5be0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 -- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0161340 msgr2=0x7f5be01f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0161340 0x7f5be01f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 -- 172.21.15.53:0/783656521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd8009a70 con 0x7f5be015c9b0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bde59c700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 0x7f5be0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd7fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c510 tx=0x7f5bc800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80052d0 con 0x7f5be015c9b0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8005430 con 0x7f5be015c9b0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.848+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be00d1300 con 0x7f5be015c9b0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.849+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80038c0 con 0x7f5be015c9b0 2023-11-28T11:18:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.849+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be00b5520 con 0x7f5be015c9b0 2023-11-28T11:18:39.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.850+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be00d15e0 con 0x7f5be015c9b0 2023-11-28T11:18:39.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.850+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5bc8019030 con 0x7f5be015c9b0 2023-11-28T11:18:39.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.851+0000 7f5bd6ffd700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 0x7f5bc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:39.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.851+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5bc8085060 con 0x7f5be015c9b0 2023-11-28T11:18:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.852+0000 7f5bde59c700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 0x7f5bc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.853+0000 7f5bde59c700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 0x7f5bc006a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0001eb0 tx=0x7f5bd0001d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:39.855+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5bc8051790 con 0x7f5be015c9b0 2023-11-28T11:18:40.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.062+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5be00745c0 con 0x7f5be015c9b0 2023-11-28T11:18:40.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.062+0000 7f5bd6ffd700 1 -- 172.21.15.53:0/783656521 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5bc80170a0 con 0x7f5be015c9b0 2023-11-28T11:18:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:40.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 msgr2=0x7f5bc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 0x7f5bc006a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0001eb0 tx=0x7f5bd0001d00 comp rx=0 tx=0).stop 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 msgr2=0x7f5be00c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c510 tx=0x7f5bc800c820 comp rx=0 tx=0).stop 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 shutdown_connections 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5bc00682f0 0x7f5bc006a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be00cd7c0 0x7f5be0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0161340 0x7f5be01f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 --2- 172.21.15.53:0/783656521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be015c9b0 0x7f5be00c9b20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 >> 172.21.15.53:0/783656521 conn(0x7f5be0072600 msgr2=0x7f5be015abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:40.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 shutdown_connections 2023-11-28T11:18:40.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.065+0000 7f5be4fff700 1 -- 172.21.15.53:0/783656521 wait complete. 2023-11-28T11:18:40.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:18:40.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.286+0000 7f6b675ff700 1 Processor -- start 2023-11-28T11:18:40.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.286+0000 7f6b675ff700 1 -- start start 2023-11-28T11:18:40.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.287+0000 7f6b675ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b6015c9b0 0x7f6b600b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b675ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b675ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b600d2320 0x7f6b600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b675ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600b5800 con 0x7f6b6015c9b0 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b675ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600d1950 con 0x7f6b600d2320 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b675ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600b5520 con 0x7f6b60161340 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b6015c9b0 0x7f6b600b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b6015c9b0 0x7f6b600b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54206/0 (socket says 172.21.15.53:54206) 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 -- 172.21.15.53:0/3181985352 learned_addr learned my addr 172.21.15.53:0/3181985352 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b64b9a700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b65b9c700 1 --2- 172.21.15.53:0/3181985352 >> v2:172.21.15.157:3300/0 conn(0x7f6b600d2320 0x7f6b600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 -- 172.21.15.53:0/3181985352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 msgr2=0x7f6b600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 -- 172.21.15.53:0/3181985352 >> v2:172.21.15.157:3300/0 conn(0x7f6b600d2320 msgr2=0x7f6b600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3181985352 >> v2:172.21.15.157:3300/0 conn(0x7f6b600d2320 0x7f6b600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.288+0000 7f6b6539b700 1 -- 172.21.15.53:0/3181985352 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600d1300 con 0x7f6b6015c9b0 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6015c9b0 0x7f6b600b4bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6b6008c4d0 tx=0x7f6b5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcce3c069c7b631 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b57fff700 1 -- 172.21.15.53:0/3181985352 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00b870 con 0x7f6b6015c9b0 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b57fff700 1 -- 172.21.15.53:0/3181985352 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00b9d0 con 0x7f6b6015c9b0 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b57fff700 1 -- 172.21.15.53:0/3181985352 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00bd60 con 0x7f6b6015c9b0 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 -- 172.21.15.53:0/3181985352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6015c9b0 msgr2=0x7f6b600b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6015c9b0 0x7f6b600b4bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6b6008c4d0 tx=0x7f6b5c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 -- 172.21.15.53:0/3181985352 shutdown_connections 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3181985352 >> v2:172.21.15.157:3300/0 conn(0x7f6b600d2320 0x7f6b600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6015c9b0 0x7f6b600b4bf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3181985352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.289+0000 7f6b675ff700 1 -- 172.21.15.53:0/3181985352 >> 172.21.15.53:0/3181985352 conn(0x7f6b60072600 msgr2=0x7f6b600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:40.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.290+0000 7f6b675ff700 1 -- 172.21.15.53:0/3181985352 shutdown_connections 2023-11-28T11:18:40.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.290+0000 7f6b675ff700 1 -- 172.21.15.53:0/3181985352 wait complete. 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.291+0000 7f6b675ff700 1 Processor -- start 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.291+0000 7f6b675ff700 1 -- start start 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 0x7f6b600cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600c9100 con 0x7f6b600d2320 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600c93e0 con 0x7f6b6015c9b0 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b675ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600cafe0 con 0x7f6b60161340 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.292+0000 7f6b6539b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54210/0 (socket says 172.21.15.53:54210) 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 -- 172.21.15.53:0/3975035728 learned_addr learned my addr 172.21.15.53:0/3975035728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:40.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b64b9a700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 0x7f6b600cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b65b9c700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 -- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 msgr2=0x7f6b600cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 -- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 msgr2=0x7f6b600cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 0x7f6b600cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 -- 172.21.15.53:0/3975035728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009550 con 0x7f6b600d2320 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b64b9a700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 0x7f6b600cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b6539b700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c004520 con 0x7f6b600d2320 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c01e070 con 0x7f6b600d2320 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0038c0 con 0x7f6b600d2320 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.293+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b600b5130 con 0x7f6b600d2320 2023-11-28T11:18:40.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.294+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b600d15e0 con 0x7f6b600d2320 2023-11-28T11:18:40.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.294+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b600b52c0 con 0x7f6b600d2320 2023-11-28T11:18:40.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.296+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b5c00f050 con 0x7f6b600d2320 2023-11-28T11:18:40.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.296+0000 7f6b55ffb700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 0x7f6b4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.296+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6b5c020070 con 0x7f6b600d2320 2023-11-28T11:18:40.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.297+0000 7f6b64b9a700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 0x7f6b4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.298+0000 7f6b64b9a700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 0x7f6b4406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b600c9ca0 tx=0x7f6b50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.301+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b5c053a60 con 0x7f6b600d2320 2023-11-28T11:18:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.519+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b600745c0 con 0x7f6b600d2320 2023-11-28T11:18:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.519+0000 7f6b55ffb700 1 -- 172.21.15.53:0/3975035728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b5c015080 con 0x7f6b600d2320 2023-11-28T11:18:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 msgr2=0x7f6b4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 0x7f6b4406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b600c9ca0 tx=0x7f6b50008040 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 msgr2=0x7f6b6014c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c004750 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 shutdown_connections 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6b440682f0 0x7f6b4406a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b60161340 0x7f6b600cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b600d2320 0x7f6b6014c470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 --2- 172.21.15.53:0/3975035728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6015c9b0 0x7f6b600cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 >> 172.21.15.53:0/3975035728 conn(0x7f6b60072600 msgr2=0x7f6b600b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.521+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 shutdown_connections 2023-11-28T11:18:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.522+0000 7f6b675ff700 1 -- 172.21.15.53:0/3975035728 wait complete. 2023-11-28T11:18:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:18:40.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T11:18:40.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:40.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T11:18:40.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:18:40.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:18:40.623 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:18:40.669 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:18:40.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.871+0000 7f92d31ff700 1 Processor -- start 2023-11-28T11:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.871+0000 7f92d31ff700 1 -- start start 2023-11-28T11:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d31ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92cc15c9b0 0x7f92cc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d31ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92cc161340 0x7f92cc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d31ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc0d1950 con 0x7f92cc0d2320 2023-11-28T11:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d31ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0b5800 con 0x7f92cc161340 2023-11-28T11:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d31ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0b5520 con 0x7f92cc15c9b0 2023-11-28T11:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92cbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92cc161340 0x7f92cc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92cbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92cc161340 0x7f92cc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46374/0 (socket says 172.21.15.53:46374) 2023-11-28T11:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92cbfff700 1 -- 172.21.15.53:0/3238261748 learned_addr learned my addr 172.21.15.53:0/3238261748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.872+0000 7f92d0f9b700 1 --2- 172.21.15.53:0/3238261748 >> v2:172.21.15.157:3300/0 conn(0x7f92cc15c9b0 0x7f92cc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cbfff700 1 -- 172.21.15.53:0/3238261748 >> v2:172.21.15.157:3300/0 conn(0x7f92cc15c9b0 msgr2=0x7f92cc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:40.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cbfff700 1 --2- 172.21.15.53:0/3238261748 >> v2:172.21.15.157:3300/0 conn(0x7f92cc15c9b0 0x7f92cc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cbfff700 1 -- 172.21.15.53:0/3238261748 >> v1:172.21.15.157:6789/0 conn(0x7f92cc0d2320 legacy=0x7f92cc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cbfff700 1 -- 172.21.15.53:0/3238261748 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0d1300 con 0x7f92cc161340 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cbfff700 1 --2- 172.21.15.53:0/3238261748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc161340 0x7f92cc0d1de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f92cc067460 tx=0x7f92b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8dd0211e816634 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cb7fe700 1 -- 172.21.15.53:0/3238261748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8011830 con 0x7f92cc161340 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cb7fe700 1 -- 172.21.15.53:0/3238261748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8011990 con 0x7f92cc161340 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.873+0000 7f92cb7fe700 1 -- 172.21.15.53:0/3238261748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8011d20 con 0x7f92cc161340 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 -- 172.21.15.53:0/3238261748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc161340 msgr2=0x7f92cc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 --2- 172.21.15.53:0/3238261748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc161340 0x7f92cc0d1de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f92cc067460 tx=0x7f92b8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 -- 172.21.15.53:0/3238261748 shutdown_connections 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 --2- 172.21.15.53:0/3238261748 >> v2:172.21.15.157:3300/0 conn(0x7f92cc15c9b0 0x7f92cc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 --2- 172.21.15.53:0/3238261748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc161340 0x7f92cc0d1de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 -- 172.21.15.53:0/3238261748 >> 172.21.15.53:0/3238261748 conn(0x7f92cc072600 msgr2=0x7f92cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:40.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 -- 172.21.15.53:0/3238261748 shutdown_connections 2023-11-28T11:18:40.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.874+0000 7f92d31ff700 1 -- 172.21.15.53:0/3238261748 wait complete. 2023-11-28T11:18:40.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.876+0000 7f92d31ff700 1 Processor -- start 2023-11-28T11:18:40.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.876+0000 7f92d31ff700 1 -- start start 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d2320 0x7f92cc143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc161340 0x7f92cc0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc153580 con 0x7f92cc15c9b0 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0c6170 con 0x7f92cc0d2320 2023-11-28T11:18:40.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d31ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0c7680 con 0x7f92cc161340 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46388/0 (socket says 172.21.15.53:46388) 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d0f9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d2320 0x7f92cc143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 -- 172.21.15.53:0/2491412270 learned_addr learned my addr 172.21.15.53:0/2491412270 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92d179c700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc161340 0x7f92cc0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 -- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc161340 msgr2=0x7f92cc0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc161340 0x7f92cc0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 -- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d2320 msgr2=0x7f92cc143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d2320 0x7f92cc143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.877+0000 7f92cbfff700 1 -- 172.21.15.53:0/2491412270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009550 con 0x7f92cc15c9b0 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92cbfff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8004480 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b801f070 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80038e0 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc0b5130 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.878+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc0d15e0 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.879+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc0b52c0 con 0x7f92cc15c9b0 2023-11-28T11:18:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.880+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92b802a020 con 0x7f92cc15c9b0 2023-11-28T11:18:40.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.881+0000 7f92c97fa700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 0x7f92b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:40.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.881+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f92b80872e0 con 0x7f92cc15c9b0 2023-11-28T11:18:40.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.882+0000 7f92d0f9b700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 0x7f92b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:40.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.882+0000 7f92d0f9b700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 0x7f92b006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009be0 tx=0x7f92c0009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:40.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:40.885+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92b8053a10 con 0x7f92cc15c9b0 2023-11-28T11:18:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.133+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92cc0745c0 con 0x7f92cc15c9b0 2023-11-28T11:18:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.134+0000 7f92c97fa700 1 -- 172.21.15.53:0/2491412270 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7f92b8057030 con 0x7f92cc15c9b0 2023-11-28T11:18:41.135 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:41.135 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 msgr2=0x7f92b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 0x7f92b006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009be0 tx=0x7f92c0009210 comp rx=0 tx=0).stop 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 msgr2=0x7f92cc0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b8004750 comp rx=0 tx=0).stop 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 shutdown_connections 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f92b00682f0 0x7f92b006a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc161340 0x7f92cc0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d2320 0x7f92cc143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 --2- 172.21.15.53:0/2491412270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92cc15c9b0 0x7f92cc0c5c30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.136+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 >> 172.21.15.53:0/2491412270 conn(0x7f92cc072600 msgr2=0x7f92cc15a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.137+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 shutdown_connections 2023-11-28T11:18:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.137+0000 7f92d31ff700 1 -- 172.21.15.53:0/2491412270 wait complete. 2023-11-28T11:18:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:18:41.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.356+0000 7ff5999ff700 1 Processor -- start 2023-11-28T11:18:41.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.356+0000 7ff5999ff700 1 -- start start 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.357+0000 7ff5999ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff594161340 0x7ff5940d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff5999ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940d1950 con 0x7ff59415c9b0 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff5999ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940b5520 con 0x7ff5940d2320 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff5999ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940b5800 con 0x7ff594161340 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff592ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff59415c9b0 0x7ff5940b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35262/0 (socket says 172.21.15.53:35262) 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff592ffd700 1 -- 172.21.15.53:0/1314434936 learned_addr learned my addr 172.21.15.53:0/1314434936 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:41.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff58bfff700 1 --2- 172.21.15.53:0/1314434936 >> v2:172.21.15.53:3300/0 conn(0x7ff594161340 0x7ff5940d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff5927fc700 1 -- 172.21.15.53:0/1314434936 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437911699 0 0) 0x7ff5940d1950 con 0x7ff59415c9b0 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff5927fc700 1 -- 172.21.15.53:0/1314434936 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5800035b0 con 0x7ff59415c9b0 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff58bfff700 1 -- 172.21.15.53:0/1314434936 >> v1:172.21.15.157:6789/0 conn(0x7ff5940d2320 legacy=0x7ff5940cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff58bfff700 1 -- 172.21.15.53:0/1314434936 >> v1:172.21.15.53:6789/0 conn(0x7ff59415c9b0 legacy=0x7ff5940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.358+0000 7ff58bfff700 1 -- 172.21.15.53:0/1314434936 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940d1300 con 0x7ff594161340 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff58bfff700 1 --2- 172.21.15.53:0/1314434936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940d1de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009840 tx=0x7ff57c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be28d441dc6df153 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff5927fc700 1 -- 172.21.15.53:0/1314434936 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011830 con 0x7ff594161340 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff5927fc700 1 -- 172.21.15.53:0/1314434936 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c011990 con 0x7ff594161340 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff5927fc700 1 -- 172.21.15.53:0/1314434936 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011d20 con 0x7ff594161340 2023-11-28T11:18:41.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff5999ff700 1 -- 172.21.15.53:0/1314434936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 msgr2=0x7ff5940d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.359+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1314434936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940d1de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009840 tx=0x7ff57c009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.360+0000 7ff5999ff700 1 -- 172.21.15.53:0/1314434936 shutdown_connections 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.360+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1314434936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940d1de0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.360+0000 7ff5999ff700 1 -- 172.21.15.53:0/1314434936 >> 172.21.15.53:0/1314434936 conn(0x7ff594072600 msgr2=0x7ff5940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.360+0000 7ff5999ff700 1 -- 172.21.15.53:0/1314434936 shutdown_connections 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.360+0000 7ff5999ff700 1 -- 172.21.15.53:0/1314434936 wait complete. 2023-11-28T11:18:41.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.361+0000 7ff5999ff700 1 Processor -- start 2023-11-28T11:18:41.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.362+0000 7ff5999ff700 1 -- start start 2023-11-28T11:18:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.362+0000 7ff5999ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5940d2320 0x7ff59414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5999ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff59415c9b0 0x7ff5941e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5999ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5999ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59414a940 con 0x7ff594161340 2023-11-28T11:18:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5999ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5941e6500 con 0x7ff59415c9b0 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5999ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5941e67e0 con 0x7ff5940d2320 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46408/0 (socket says 172.21.15.53:46408) 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 -- 172.21.15.53:0/1642040434 learned_addr learned my addr 172.21.15.53:0/1642040434 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff58bfff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff59415c9b0 0x7ff5941e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff592ffd700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5940d2320 0x7ff59414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 -- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5940d2320 msgr2=0x7ff59414a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5940d2320 0x7ff59414a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 -- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff59415c9b0 msgr2=0x7ff5941e75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff59415c9b0 0x7ff5941e75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.363+0000 7ff5937fe700 1 -- 172.21.15.53:0/1642040434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c0094a0 con 0x7ff594161340 2023-11-28T11:18:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff5937fe700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00cbc0 tx=0x7ff58c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003c00 con 0x7ff594161340 2023-11-28T11:18:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c003d60 con 0x7ff594161340 2023-11-28T11:18:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59415cd80 con 0x7ff594161340 2023-11-28T11:18:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00aae0 con 0x7ff594161340 2023-11-28T11:18:41.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.364+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940d15e0 con 0x7ff594161340 2023-11-28T11:18:41.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.365+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59415cf10 con 0x7ff594161340 2023-11-28T11:18:41.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.366+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff58c010050 con 0x7ff594161340 2023-11-28T11:18:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.367+0000 7ff590ff9700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 0x7ff57406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.367+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff58c085c10 con 0x7ff594161340 2023-11-28T11:18:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.367+0000 7ff592ffd700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 0x7ff57406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.368+0000 7ff592ffd700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 0x7ff57406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff584007fd0 tx=0x7ff584007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.371+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff58c052340 con 0x7ff594161340 2023-11-28T11:18:41.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.573+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5940745c0 con 0x7ff594161340 2023-11-28T11:18:41.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.575+0000 7ff590ff9700 1 -- 172.21.15.53:0/1642040434 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18002 (secure 0 0 0) 0x7ff58c055960 con 0x7ff594161340 2023-11-28T11:18:41.577 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:41.577 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:36.870456+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.9","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:41.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 msgr2=0x7ff57406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 0x7ff57406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff584007fd0 tx=0x7ff584007dc0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 msgr2=0x7ff5940b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00cbc0 tx=0x7ff58c00ced0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 shutdown_connections 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff5740682f0 0x7ff57406a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5940d2320 0x7ff59414a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff594161340 0x7ff5940b7970 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 --2- 172.21.15.53:0/1642040434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff59415c9b0 0x7ff5941e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 >> 172.21.15.53:0/1642040434 conn(0x7ff594072600 msgr2=0x7ff59415fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 shutdown_connections 2023-11-28T11:18:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.577+0000 7ff5999ff700 1 -- 172.21.15.53:0/1642040434 wait complete. 2023-11-28T11:18:41.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:17:47.656143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:17:48.158158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:41.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.802+0000 7fbe2d5ff700 1 Processor -- start 2023-11-28T11:18:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.802+0000 7fbe2d5ff700 1 -- start start 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe2815c9b0 0x7fbe280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28161340 0x7fbe280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe280b5800 con 0x7fbe280d2320 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe280d1950 con 0x7fbe2815c9b0 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.803+0000 7fbe2d5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280b5520 con 0x7fbe28161340 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe277fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe2815c9b0 0x7fbe280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe277fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46424/0 (socket says 172.21.15.53:46424) 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe277fe700 1 -- 172.21.15.53:0/1884248608 learned_addr learned my addr 172.21.15.53:0/1884248608 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe267fc700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28161340 0x7fbe280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 -- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28161340 msgr2=0x7fbe280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28161340 0x7fbe280d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 -- 172.21.15.53:0/1884248608 >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 msgr2=0x7fbe280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/1884248608 >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 -- 172.21.15.53:0/1884248608 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280d1300 con 0x7fbe2815c9b0 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe277fe700 1 --2- 172.21.15.53:0/1884248608 >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.804+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe280b4bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe28067460 tx=0x7fbe18009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74938e7b34c9862 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe25ffb700 1 -- 172.21.15.53:0/1884248608 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011830 con 0x7fbe2815c9b0 2023-11-28T11:18:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe25ffb700 1 -- 172.21.15.53:0/1884248608 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18011990 con 0x7fbe2815c9b0 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe25ffb700 1 -- 172.21.15.53:0/1884248608 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011d20 con 0x7fbe2815c9b0 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 msgr2=0x7fbe280b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe280b4bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe28067460 tx=0x7fbe18009b20 comp rx=0 tx=0).stop 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/1884248608 shutdown_connections 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/1884248608 >> v2:172.21.15.53:3300/0 conn(0x7fbe280d2320 0x7fbe280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe280b4bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/1884248608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe28161340 0x7fbe280d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.805+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/1884248608 >> 172.21.15.53:0/1884248608 conn(0x7fbe28072600 msgr2=0x7fbe280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.806+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/1884248608 shutdown_connections 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.806+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/1884248608 wait complete. 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.807+0000 7fbe2d5ff700 1 Processor -- start 2023-11-28T11:18:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 -- start start 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280d2320 0x7fbe28158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe281527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28154af0 con 0x7fbe28161340 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28155670 con 0x7fbe2815c9b0 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.808+0000 7fbe2d5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28154d40 con 0x7fbe280d2320 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46426/0 (socket says 172.21.15.53:46426) 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 -- 172.21.15.53:0/545773545 learned_addr learned my addr 172.21.15.53:0/545773545 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280d2320 0x7fbe28158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe267fc700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe281527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 -- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280d2320 msgr2=0x7fbe28158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280d2320 0x7fbe28158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 -- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 msgr2=0x7fbe281527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe281527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 -- 172.21.15.53:0/545773545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20009b30 con 0x7fbe28161340 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe267fc700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe281527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.809+0000 7fbe277fe700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000efc0 tx=0x7fbe2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.810+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20018070 con 0x7fbe28161340 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.810+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000f040 con 0x7fbe28161340 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.810+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe180094a0 con 0x7fbe28161340 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.810+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20003c40 con 0x7fbe28161340 2023-11-28T11:18:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.810+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe280b5130 con 0x7fbe28161340 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.811+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe280d15e0 con 0x7fbe28161340 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.812+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe2000c930 con 0x7fbe28161340 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.813+0000 7fbe0f7fe700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 0x7fbe1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.813+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbe20085ff0 con 0x7fbe28161340 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.813+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 0x7fbe1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.814+0000 7fbe26ffd700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 0x7fbe1406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe180090b0 tx=0x7fbe1801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:41.817+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe20052760 con 0x7fbe28161340 2023-11-28T11:18:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.020+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe280b52c0 con 0x7fbe28161340 2023-11-28T11:18:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.021+0000 7fbe0f7fe700 1 -- 172.21.15.53:0/545773545 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe20009ce0 con 0x7fbe28161340 2023-11-28T11:18:42.022 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 msgr2=0x7fbe1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 0x7fbe1406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe180090b0 tx=0x7fbe1801c040 comp rx=0 tx=0).stop 2023-11-28T11:18:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 msgr2=0x7fbe28152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000efc0 tx=0x7fbe2000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 shutdown_connections 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbe140682f0 0x7fbe1406a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe280d2320 0x7fbe28158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe28161340 0x7fbe28152d20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 --2- 172.21.15.53:0/545773545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe2815c9b0 0x7fbe281527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 >> 172.21.15.53:0/545773545 conn(0x7fbe28072600 msgr2=0x7fbe2815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 shutdown_connections 2023-11-28T11:18:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.023+0000 7fbe2d5ff700 1 -- 172.21.15.53:0/545773545 wait complete. 2023-11-28T11:18:42.042 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:42.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.246+0000 7fad9b5ff700 1 Processor -- start 2023-11-28T11:18:42.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.246+0000 7fad9b5ff700 1 -- start start 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad9b5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad9415c9b0 0x7fad940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad9b5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad940d1950 con 0x7fad94161340 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad9b5ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad940d24f0 con 0x7fad940cd7c0 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad9b5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad940b5800 con 0x7fad9415c9b0 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad98b9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fad94161340 0x7fad940d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35266/0 (socket says 172.21.15.53:35266) 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad9939b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad9415c9b0 0x7fad940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.247+0000 7fad98b9a700 1 -- 172.21.15.53:0/3049185262 learned_addr learned my addr 172.21.15.53:0/3049185262 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad9939b700 1 -- 172.21.15.53:0/3049185262 >> v1:172.21.15.157:6789/0 conn(0x7fad940cd7c0 legacy=0x7fad940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad8bfff700 1 -- 172.21.15.53:0/3049185262 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194647530 0 0) 0x7fad940d1950 con 0x7fad94161340 2023-11-28T11:18:42.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad9939b700 1 -- 172.21.15.53:0/3049185262 >> v1:172.21.15.53:6789/0 conn(0x7fad94161340 legacy=0x7fad940d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad9939b700 1 -- 172.21.15.53:0/3049185262 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940b5520 con 0x7fad9415c9b0 2023-11-28T11:18:42.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad9939b700 1 --2- 172.21.15.53:0/3049185262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad9415c9b0 0x7fad940b4bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad90009840 tx=0x7fad90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c6dfb8b7f9b92a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad8bfff700 1 -- 172.21.15.53:0/3049185262 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90011830 con 0x7fad9415c9b0 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad8bfff700 1 -- 172.21.15.53:0/3049185262 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90011990 con 0x7fad9415c9b0 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.248+0000 7fad8bfff700 1 -- 172.21.15.53:0/3049185262 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90011d20 con 0x7fad9415c9b0 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 -- 172.21.15.53:0/3049185262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad9415c9b0 msgr2=0x7fad940b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/3049185262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad9415c9b0 0x7fad940b4bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad90009840 tx=0x7fad90009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 -- 172.21.15.53:0/3049185262 shutdown_connections 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/3049185262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad9415c9b0 0x7fad940b4bf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 -- 172.21.15.53:0/3049185262 >> 172.21.15.53:0/3049185262 conn(0x7fad94072600 msgr2=0x7fad940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 -- 172.21.15.53:0/3049185262 shutdown_connections 2023-11-28T11:18:42.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.249+0000 7fad9b5ff700 1 -- 172.21.15.53:0/3049185262 wait complete. 2023-11-28T11:18:42.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.251+0000 7fad9b5ff700 1 Processor -- start 2023-11-28T11:18:42.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.251+0000 7fad9b5ff700 1 -- start start 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad9415c9b0 0x7fad940c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 0x7fad940b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941e8d00 con 0x7fad940cd7c0 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941e80f0 con 0x7fad94161340 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9b5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad941e83d0 con 0x7fad9415c9b0 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46440/0 (socket says 172.21.15.53:46440) 2023-11-28T11:18:42.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 -- 172.21.15.53:0/1933092947 learned_addr learned my addr 172.21.15.53:0/1933092947 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad99b9c700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 0x7fad940b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad98b9a700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad9415c9b0 0x7fad940c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 -- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad9415c9b0 msgr2=0x7fad940c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad9415c9b0 0x7fad940c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 -- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 msgr2=0x7fad940b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 0x7fad940b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad9939b700 1 -- 172.21.15.53:0/1933092947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900094a0 con 0x7fad940cd7c0 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.252+0000 7fad99b9c700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 0x7fad940b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad9939b700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad90014040 tx=0x7fad90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9001a900 con 0x7fad940cd7c0 2023-11-28T11:18:42.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9001aa60 con 0x7fad940cd7c0 2023-11-28T11:18:42.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad940d15e0 con 0x7fad940cd7c0 2023-11-28T11:18:42.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9001add0 con 0x7fad940cd7c0 2023-11-28T11:18:42.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.253+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad940d1300 con 0x7fad940cd7c0 2023-11-28T11:18:42.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.254+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad940745c0 con 0x7fad940cd7c0 2023-11-28T11:18:42.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.255+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad9002b030 con 0x7fad940cd7c0 2023-11-28T11:18:42.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.256+0000 7fad89ffb700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 0x7fad7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.256+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fad90087140 con 0x7fad940cd7c0 2023-11-28T11:18:42.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.257+0000 7fad98b9a700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 0x7fad7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.257+0000 7fad98b9a700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 0x7fad7806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad84007fd0 tx=0x7fad84007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.260+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad90053960 con 0x7fad940cd7c0 2023-11-28T11:18:42.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.461+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad940d08a0 con 0x7fad940cd7c0 2023-11-28T11:18:42.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.462+0000 7fad89ffb700 1 -- 172.21.15.53:0/1933092947 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad90015020 con 0x7fad940cd7c0 2023-11-28T11:18:42.463 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:42.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 msgr2=0x7fad7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 0x7fad7806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad84007fd0 tx=0x7fad84007c20 comp rx=0 tx=0).stop 2023-11-28T11:18:42.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 msgr2=0x7fad9414a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad90014040 tx=0x7fad90004750 comp rx=0 tx=0).stop 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 shutdown_connections 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fad780682f0 0x7fad7806a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad94161340 0x7fad940b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad940cd7c0 0x7fad9414a5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 --2- 172.21.15.53:0/1933092947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad9415c9b0 0x7fad940c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 >> 172.21.15.53:0/1933092947 conn(0x7fad94072600 msgr2=0x7fad9415a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 shutdown_connections 2023-11-28T11:18:42.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.464+0000 7fad9b5ff700 1 -- 172.21.15.53:0/1933092947 wait complete. 2023-11-28T11:18:42.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:18:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.687+0000 7f1fec7ff700 1 Processor -- start 2023-11-28T11:18:42.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.687+0000 7f1fec7ff700 1 -- start start 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe40d01b0 0x7f1fe40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 0x7f1fe40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b29b0 0x7f1fe40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe415cec0 con 0x7f1fe40d01b0 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40b0230 con 0x7f1fe40b20b0 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fec7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40b4570 con 0x7f1fe40b29b0 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fea59b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe40d01b0 0x7f1fe40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fea59b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe40d01b0 0x7f1fe40d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46452/0 (socket says 172.21.15.53:46452) 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.688+0000 7f1fea59b700 1 -- 172.21.15.53:0/2709724536 learned_addr learned my addr 172.21.15.53:0/2709724536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fead9c700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b29b0 0x7f1fe40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fe9d9a700 1 --2- 172.21.15.53:0/2709724536 >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 0x7f1fe40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 -- 172.21.15.53:0/2709724536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b29b0 msgr2=0x7f1fe40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b29b0 0x7f1fe40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 -- 172.21.15.53:0/2709724536 >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 msgr2=0x7f1fe40d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 --2- 172.21.15.53:0/2709724536 >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 0x7f1fe40d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 -- 172.21.15.53:0/2709724536 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40b4850 con 0x7f1fe40d01b0 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fe9d9a700 1 --2- 172.21.15.53:0/2709724536 >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 0x7f1fe40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fea59b700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40d0c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40ca0b0 tx=0x7f1fe0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5b957d9bfc314e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fe9599700 1 -- 172.21.15.53:0/2709724536 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000b870 con 0x7f1fe40d01b0 2023-11-28T11:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fe9599700 1 -- 172.21.15.53:0/2709724536 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe000b9d0 con 0x7f1fe40d01b0 2023-11-28T11:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.689+0000 7f1fe9599700 1 -- 172.21.15.53:0/2709724536 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000bd60 con 0x7f1fe40d01b0 2023-11-28T11:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 -- 172.21.15.53:0/2709724536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 msgr2=0x7f1fe40d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40d0c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40ca0b0 tx=0x7f1fe0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 -- 172.21.15.53:0/2709724536 shutdown_connections 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b29b0 0x7f1fe40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/2709724536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40d0c20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/2709724536 >> v2:172.21.15.157:3300/0 conn(0x7f1fe40b20b0 0x7f1fe40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 -- 172.21.15.53:0/2709724536 >> 172.21.15.53:0/2709724536 conn(0x7f1fe409c3f0 msgr2=0x7f1fe40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 -- 172.21.15.53:0/2709724536 shutdown_connections 2023-11-28T11:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.690+0000 7f1fec7ff700 1 -- 172.21.15.53:0/2709724536 wait complete. 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.692+0000 7f1fec7ff700 1 Processor -- start 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.692+0000 7f1fec7ff700 1 -- start start 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b20b0 0x7f1fe40b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe40b29b0 0x7f1fe40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40ca660 con 0x7f1fe40d01b0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40ca890 con 0x7f1fe40b29b0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fec7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40cab70 con 0x7f1fe40b20b0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fead9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fead9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46460/0 (socket says 172.21.15.53:46460) 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fea59b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b20b0 0x7f1fe40b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fead9c700 1 -- 172.21.15.53:0/1755033058 learned_addr learned my addr 172.21.15.53:0/1755033058 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.693+0000 7f1fe9d9a700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe40b29b0 0x7f1fe40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 -- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b20b0 msgr2=0x7f1fe40b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b20b0 0x7f1fe40b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 -- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe40b29b0 msgr2=0x7f1fe40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe40b29b0 0x7f1fe40cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 -- 172.21.15.53:0/1755033058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0009550 con 0x7f1fe40d01b0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fead9c700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc007e40 tx=0x7f1fdc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ed30 con 0x7f1fe40d01b0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc014420 con 0x7f1fe40d01b0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe40afec0 con 0x7f1fe40d01b0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0147d0 con 0x7f1fe40d01b0 2023-11-28T11:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.694+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe40afbe0 con 0x7f1fe40d01b0 2023-11-28T11:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.695+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe40b2600 con 0x7f1fe40d01b0 2023-11-28T11:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.696+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1fdc012050 con 0x7f1fe40d01b0 2023-11-28T11:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.697+0000 7f1fdaffd700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 0x7f1fc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.697+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1fdc086340 con 0x7f1fe40d01b0 2023-11-28T11:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.697+0000 7f1fea59b700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 0x7f1fc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:42.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.698+0000 7f1fea59b700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 0x7f1fc806a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014040 tx=0x7f1fe0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.701+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1fdc052a70 con 0x7f1fe40d01b0 2023-11-28T11:18:42.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.904+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1fe4068690 con 0x7f1fe40d01b0 2023-11-28T11:18:42.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.904+0000 7f1fdaffd700 1 -- 172.21.15.53:0/1755033058 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1fdc056090 con 0x7f1fe40d01b0 2023-11-28T11:18:42.905 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.906+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 msgr2=0x7f1fc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.906+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 0x7f1fc806a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014040 tx=0x7f1fe0008030 comp rx=0 tx=0).stop 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.906+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 msgr2=0x7f1fe40ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.906+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc007e40 tx=0x7f1fdc00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 shutdown_connections 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1fc80682f0 0x7f1fc806a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe40b20b0 0x7f1fe40b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe40d01b0 0x7f1fe40ca120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 --2- 172.21.15.53:0/1755033058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe40b29b0 0x7f1fe40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:42.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 >> 172.21.15.53:0/1755033058 conn(0x7f1fe409c3f0 msgr2=0x7f1fe40d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:42.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 shutdown_connections 2023-11-28T11:18:42.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:42.907+0000 7f1fec7ff700 1 -- 172.21.15.53:0/1755033058 wait complete. 2023-11-28T11:18:42.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:18:43.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.129+0000 7fb87bdff700 1 Processor -- start 2023-11-28T11:18:43.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.130+0000 7fb87bdff700 1 -- start start 2023-11-28T11:18:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.131+0000 7fb87bdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87415c9b0 0x7fb8740b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.131+0000 7fb87bdff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb8740cd7c0 0x7fb8740cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.131+0000 7fb87bdff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740ce160 con 0x7fb874161340 2023-11-28T11:18:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.131+0000 7fb87bdff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8740b5800 con 0x7fb8740cd7c0 2023-11-28T11:18:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.131+0000 7fb87bdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740d1950 con 0x7fb87415c9b0 2023-11-28T11:18:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87939a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb874161340 0x7fb8740d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35274/0 (socket says 172.21.15.53:35274) 2023-11-28T11:18:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb8740cd7c0 0x7fb8740cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87939a700 1 -- 172.21.15.53:0/2267411332 learned_addr learned my addr 172.21.15.53:0/2267411332 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb879b9b700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87415c9b0 0x7fb8740b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 -- 172.21.15.53:0/2267411332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87415c9b0 msgr2=0x7fb8740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87415c9b0 0x7fb8740b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 -- 172.21.15.53:0/2267411332 >> v1:172.21.15.53:6789/0 conn(0x7fb874161340 legacy=0x7fb8740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 -- 172.21.15.53:0/2267411332 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740d24f0 con 0x7fb8740cd7c0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb878b99700 1 -- 172.21.15.53:0/2267411332 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855207830 0 0) 0x7fb8740ce160 con 0x7fb874161340 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.132+0000 7fb87a39c700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8740cd7c0 0x7fb8740cdc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009d90 tx=0x7fb86c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ad8ad53037263f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb878b99700 1 -- 172.21.15.53:0/2267411332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00e070 con 0x7fb8740cd7c0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb878b99700 1 -- 172.21.15.53:0/2267411332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00bbd0 con 0x7fb8740cd7c0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb878b99700 1 -- 172.21.15.53:0/2267411332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c012430 con 0x7fb8740cd7c0 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 -- 172.21.15.53:0/2267411332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8740cd7c0 msgr2=0x7fb8740cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8740cd7c0 0x7fb8740cdc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009d90 tx=0x7fb86c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 -- 172.21.15.53:0/2267411332 shutdown_connections 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8740cd7c0 0x7fb8740cdc20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 --2- 172.21.15.53:0/2267411332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87415c9b0 0x7fb8740b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 -- 172.21.15.53:0/2267411332 >> 172.21.15.53:0/2267411332 conn(0x7fb874072600 msgr2=0x7fb8740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 -- 172.21.15.53:0/2267411332 shutdown_connections 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.133+0000 7fb87bdff700 1 -- 172.21.15.53:0/2267411332 wait complete. 2023-11-28T11:18:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.136+0000 7fb87bdff700 1 Processor -- start 2023-11-28T11:18:43.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.136+0000 7fb87bdff700 1 -- start start 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8740cd7c0 0x7fb8741f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb874161340 0x7fb87415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87415f440 con 0x7fb87415c9b0 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87415eba0 con 0x7fb8740cd7c0 2023-11-28T11:18:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87bdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87415ee80 con 0x7fb874161340 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.137+0000 7fb87939a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46476/0 (socket says 172.21.15.53:46476) 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 -- 172.21.15.53:0/1140093748 learned_addr learned my addr 172.21.15.53:0/1140093748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 -- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb874161340 msgr2=0x7fb87415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb874161340 0x7fb87415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 -- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8740cd7c0 msgr2=0x7fb8741f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8740cd7c0 0x7fb8741f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 -- 172.21.15.53:0/1140093748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c009a70 con 0x7fb87415c9b0 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87939a700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c550 tx=0x7fb86400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864004c80 con 0x7fb87415c9b0 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864004e00 con 0x7fb87415c9b0 2023-11-28T11:18:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8740d1300 con 0x7fb87415c9b0 2023-11-28T11:18:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.138+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864007310 con 0x7fb87415c9b0 2023-11-28T11:18:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.139+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8740b5520 con 0x7fb87415c9b0 2023-11-28T11:18:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.139+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8740d15e0 con 0x7fb87415c9b0 2023-11-28T11:18:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.140+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb86401a020 con 0x7fb87415c9b0 2023-11-28T11:18:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.141+0000 7fb86a7fc700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 0x7fb85806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.141+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb8640858e0 con 0x7fb87415c9b0 2023-11-28T11:18:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.142+0000 7fb879b9b700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 0x7fb85806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.143+0000 7fb879b9b700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 0x7fb85806a7f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8700098d0 tx=0x7fb870008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.146+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb864052090 con 0x7fb87415c9b0 2023-11-28T11:18:43.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.353+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8740745c0 con 0x7fb87415c9b0 2023-11-28T11:18:43.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.353+0000 7fb86a7fc700 1 -- 172.21.15.53:0/1140093748 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8640556b0 con 0x7fb87415c9b0 2023-11-28T11:18:43.354 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.355+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 msgr2=0x7fb85806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.355+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 0x7fb85806a7f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8700098d0 tx=0x7fb870008040 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.355+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 msgr2=0x7fb87415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.355+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c550 tx=0x7fb86400c910 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 shutdown_connections 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb858068340 0x7fb85806a7f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb874161340 0x7fb87415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8740cd7c0 0x7fb8741f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 --2- 172.21.15.53:0/1140093748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb87415c9b0 0x7fb87415e2c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 >> 172.21.15.53:0/1140093748 conn(0x7fb874072600 msgr2=0x7fb8740d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:43.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 shutdown_connections 2023-11-28T11:18:43.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.356+0000 7fb87bdff700 1 -- 172.21.15.53:0/1140093748 wait complete. 2023-11-28T11:18:43.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:18:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.576+0000 7f8cc59ff700 1 Processor -- start 2023-11-28T11:18:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.576+0000 7f8cc59ff700 1 -- start start 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cc59ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cc59ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8cc00b20b0 0x7f8cc00d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cc59ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc00b0230 con 0x7f8cc00b29b0 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cc59ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015cec0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cc59ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b4570 con 0x7f8cc00d01b0 2023-11-28T11:18:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cb7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8cc00b20b0 0x7f8cc00d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cb7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8cc00b20b0 0x7f8cc00d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46478/0 (socket says 172.21.15.53:46478) 2023-11-28T11:18:43.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cb7fff700 1 -- 172.21.15.53:0/522536321 learned_addr learned my addr 172.21.15.53:0/522536321 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:43.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.577+0000 7f8cbeffd700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cb7fff700 1 -- 172.21.15.53:0/522536321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 msgr2=0x7f8cc00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:43.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cb7fff700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc00d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cb7fff700 1 -- 172.21.15.53:0/522536321 >> v1:172.21.15.157:6789/0 conn(0x7f8cc00b29b0 legacy=0x7f8cc00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cb7fff700 1 -- 172.21.15.53:0/522536321 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00b4850 con 0x7f8cc00b20b0 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cb7fff700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00d1160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80098f0 tx=0x7f8ca8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec64d487edeacbee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cbe7fc700 1 -- 172.21.15.53:0/522536321 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011830 con 0x7f8cc00b20b0 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cbe7fc700 1 -- 172.21.15.53:0/522536321 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8011990 con 0x7f8cc00b20b0 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.578+0000 7f8cbe7fc700 1 -- 172.21.15.53:0/522536321 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011d20 con 0x7f8cc00b20b0 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 -- 172.21.15.53:0/522536321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 msgr2=0x7f8cc00d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00d1160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80098f0 tx=0x7f8ca8009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 -- 172.21.15.53:0/522536321 shutdown_connections 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc00d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/522536321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00d1160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 -- 172.21.15.53:0/522536321 >> 172.21.15.53:0/522536321 conn(0x7f8cc009c3f0 msgr2=0x7f8cc00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 -- 172.21.15.53:0/522536321 shutdown_connections 2023-11-28T11:18:43.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.579+0000 7f8cc59ff700 1 -- 172.21.15.53:0/522536321 wait complete. 2023-11-28T11:18:43.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.581+0000 7f8cc59ff700 1 Processor -- start 2023-11-28T11:18:43.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.581+0000 7f8cc59ff700 1 -- start start 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc00b29b0 0x7f8cc00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc01ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00cc7b0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00cd400 con 0x7f8cc00b29b0 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cc59ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01f32a0 con 0x7f8cc00d01b0 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46490/0 (socket says 172.21.15.53:46490) 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 -- 172.21.15.53:0/1404841657 learned_addr learned my addr 172.21.15.53:0/1404841657 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cb7fff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc00b29b0 0x7f8cc00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbf7fe700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc01ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 -- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 msgr2=0x7f8cc01ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc01ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 -- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc00b29b0 msgr2=0x7f8cc00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc00b29b0 0x7f8cc00cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.582+0000 7f8cbeffd700 1 -- 172.21.15.53:0/1404841657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009550 con 0x7f8cc00b20b0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cbeffd700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000e870 tx=0x7f8cb000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0009590 con 0x7f8cc00b20b0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0013680 con 0x7f8cc00b20b0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00139f0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc00afec0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.583+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc00afbe0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.584+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc00b2600 con 0x7f8cc00b20b0 2023-11-28T11:18:43.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.585+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8cb0019030 con 0x7f8cc00b20b0 2023-11-28T11:18:43.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.586+0000 7f8cbcff9700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:43.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.586+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8cb0085a80 con 0x7f8cc00b20b0 2023-11-28T11:18:43.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.586+0000 7f8cb7fff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:43.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.587+0000 7f8cb7fff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 0x7f8ca006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009c40 tx=0x7f8ca8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:43.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.590+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8cb00521b0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.841+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc00cc0c0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.841+0000 7f8cbcff9700 1 -- 172.21.15.53:0/1404841657 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7f8cb00557d0 con 0x7f8cc00b20b0 2023-11-28T11:18:43.842 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:43.843 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 msgr2=0x7f8ca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 0x7f8ca006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009c40 tx=0x7f8ca8008030 comp rx=0 tx=0).stop 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 msgr2=0x7f8cc00b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000e870 tx=0x7f8cb000eb80 comp rx=0 tx=0).stop 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 shutdown_connections 2023-11-28T11:18:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cc00d01b0 0x7f8cc01ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc00b20b0 0x7f8cc00b8810 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 --2- 172.21.15.53:0/1404841657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cc00b29b0 0x7f8cc00cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 >> 172.21.15.53:0/1404841657 conn(0x7f8cc009c3f0 msgr2=0x7f8cc00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 shutdown_connections 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:43.844+0000 7f8cc59ff700 1 -- 172.21.15.53:0/1404841657 wait complete. 2023-11-28T11:18:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:18:43.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:44.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.065+0000 7f23899ff700 1 Processor -- start 2023-11-28T11:18:44.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.065+0000 7f23899ff700 1 -- start start 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23899ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238415c9b0 0x7f23840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23899ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f23840cd7c0 0x7f23840cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23899ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23840ce160 con 0x7f2384161340 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23899ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840b5800 con 0x7f23840cd7c0 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23899ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23840d1950 con 0x7f238415c9b0 2023-11-28T11:18:44.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f23837fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f23840cd7c0 0x7f23840cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f237bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2384161340 0x7f23840d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35276/0 (socket says 172.21.15.53:35276) 2023-11-28T11:18:44.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f237bfff700 1 -- 172.21.15.53:0/1519469456 learned_addr learned my addr 172.21.15.53:0/1519469456 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:44.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.066+0000 7f2382ffd700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238415c9b0 0x7f23840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23837fe700 1 -- 172.21.15.53:0/1519469456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238415c9b0 msgr2=0x7f23840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:44.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23837fe700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238415c9b0 0x7f23840b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23837fe700 1 -- 172.21.15.53:0/1519469456 >> v1:172.21.15.53:6789/0 conn(0x7f2384161340 legacy=0x7f23840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23837fe700 1 -- 172.21.15.53:0/1519469456 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840d24f0 con 0x7f23840cd7c0 2023-11-28T11:18:44.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23827fc700 1 -- 172.21.15.53:0/1519469456 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702739524 0 0) 0x7f23840ce160 con 0x7f2384161340 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23837fe700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f23840cdc20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f237c009d90 tx=0x7f237c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb193cffdf8edbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23827fc700 1 -- 172.21.15.53:0/1519469456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c00e070 con 0x7f23840cd7c0 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23827fc700 1 -- 172.21.15.53:0/1519469456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c00bbd0 con 0x7f23840cd7c0 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.067+0000 7f23827fc700 1 -- 172.21.15.53:0/1519469456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c012430 con 0x7f23840cd7c0 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 -- 172.21.15.53:0/1519469456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 msgr2=0x7f23840cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f23840cdc20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f237c009d90 tx=0x7f237c00b330 comp rx=0 tx=0).stop 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 -- 172.21.15.53:0/1519469456 shutdown_connections 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f23840cdc20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 --2- 172.21.15.53:0/1519469456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238415c9b0 0x7f23840b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 -- 172.21.15.53:0/1519469456 >> 172.21.15.53:0/1519469456 conn(0x7f2384072600 msgr2=0x7f23840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 -- 172.21.15.53:0/1519469456 shutdown_connections 2023-11-28T11:18:44.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.068+0000 7f23899ff700 1 -- 172.21.15.53:0/1519469456 wait complete. 2023-11-28T11:18:44.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.070+0000 7f23899ff700 1 Processor -- start 2023-11-28T11:18:44.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.070+0000 7f23899ff700 1 -- start start 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238415c9b0 0x7f23840c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2384161340 0x7f23841f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841e83f0 con 0x7f23840cd7c0 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841e77e0 con 0x7f238415c9b0 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23899ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23841e7ac0 con 0x7f2384161340 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46506/0 (socket says 172.21.15.53:46506) 2023-11-28T11:18:44.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 -- 172.21.15.53:0/1143131383 learned_addr learned my addr 172.21.15.53:0/1143131383 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f237bfff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238415c9b0 0x7f23840c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f23837fe700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2384161340 0x7f23841f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 -- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2384161340 msgr2=0x7f23841f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2384161340 0x7f23841f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 -- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238415c9b0 msgr2=0x7f23840c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238415c9b0 0x7f23840c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.071+0000 7f2382ffd700 1 -- 172.21.15.53:0/1143131383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009a70 con 0x7f23840cd7c0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f2382ffd700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f237400d800 tx=0x7f237400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740125f0 con 0x7f23840cd7c0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374012750 con 0x7f23840cd7c0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374012b00 con 0x7f23840cd7c0 2023-11-28T11:18:44.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23840d1300 con 0x7f23840cd7c0 2023-11-28T11:18:44.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.072+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23840b5520 con 0x7f23840cd7c0 2023-11-28T11:18:44.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.073+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23840d15e0 con 0x7f23840cd7c0 2023-11-28T11:18:44.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.074+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2374012c60 con 0x7f23840cd7c0 2023-11-28T11:18:44.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.075+0000 7f2380ff9700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 0x7f236406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.075+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2374086260 con 0x7f23840cd7c0 2023-11-28T11:18:44.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.075+0000 7f237bfff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 0x7f236406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.076+0000 7f237bfff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 0x7f236406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f238415fac0 tx=0x7f236c007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.079+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2374052990 con 0x7f23840cd7c0 2023-11-28T11:18:44.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.280+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23841e86d0 con 0x7f23840cd7c0 2023-11-28T11:18:44.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.282+0000 7f2380ff9700 1 -- 172.21.15.53:0/1143131383 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18002 (secure 0 0 0) 0x7f2374055fb0 con 0x7f23840cd7c0 2023-11-28T11:18:44.283 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:44.283 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:36.870456+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.9","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:44.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 msgr2=0x7f236406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 0x7f236406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f238415fac0 tx=0x7f236c007c90 comp rx=0 tx=0).stop 2023-11-28T11:18:44.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 msgr2=0x7f238414a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f237400d800 tx=0x7f237400dbc0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 shutdown_connections 2023-11-28T11:18:44.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f23640682f0 0x7f236406a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2384161340 0x7f23841f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23840cd7c0 0x7f238414a5f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 --2- 172.21.15.53:0/1143131383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238415c9b0 0x7f23840c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 >> 172.21.15.53:0/1143131383 conn(0x7f2384072600 msgr2=0x7f23840b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 shutdown_connections 2023-11-28T11:18:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.284+0000 7f23899ff700 1 -- 172.21.15.53:0/1143131383 wait complete. 2023-11-28T11:18:44.304 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:18:44.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:18:44.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.504+0000 7f97359ff700 1 Processor -- start 2023-11-28T11:18:44.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.504+0000 7f97359ff700 1 -- start start 2023-11-28T11:18:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f97359ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973015c9b0 0x7f97300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f97359ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9730161340 0x7f97300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f97359ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97300d1950 con 0x7f97300d2320 2023-11-28T11:18:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f97359ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300b5800 con 0x7f973015c9b0 2023-11-28T11:18:44.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f97359ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97300b5520 con 0x7f9730161340 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973015c9b0 0x7f97300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973015c9b0 0x7f97300b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46516/0 (socket says 172.21.15.53:46516) 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 -- 172.21.15.53:0/104526256 learned_addr learned my addr 172.21.15.53:0/104526256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f9727fff700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9730161340 0x7f97300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 -- 172.21.15.53:0/104526256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9730161340 msgr2=0x7f97300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9730161340 0x7f97300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 -- 172.21.15.53:0/104526256 >> v1:172.21.15.157:6789/0 conn(0x7f97300d2320 legacy=0x7f97300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.506+0000 7f972effd700 1 -- 172.21.15.53:0/104526256 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300d1300 con 0x7f973015c9b0 2023-11-28T11:18:44.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:18:44.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.507+0000 7f972effd700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973015c9b0 0x7f97300b4bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97200098f0 tx=0x7f9720009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f088e281be233 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.507+0000 7f972e7fc700 1 -- 172.21.15.53:0/104526256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000b8a0 con 0x7f973015c9b0 2023-11-28T11:18:44.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.507+0000 7f972e7fc700 1 -- 172.21.15.53:0/104526256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972000ba00 con 0x7f973015c9b0 2023-11-28T11:18:44.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.507+0000 7f972e7fc700 1 -- 172.21.15.53:0/104526256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000bd90 con 0x7f973015c9b0 2023-11-28T11:18:44.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 -- 172.21.15.53:0/104526256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973015c9b0 msgr2=0x7f97300b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973015c9b0 0x7f97300b4bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97200098f0 tx=0x7f9720009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 -- 172.21.15.53:0/104526256 shutdown_connections 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973015c9b0 0x7f97300b4bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 --2- 172.21.15.53:0/104526256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9730161340 0x7f97300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 -- 172.21.15.53:0/104526256 >> 172.21.15.53:0/104526256 conn(0x7f9730072600 msgr2=0x7f97300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 -- 172.21.15.53:0/104526256 shutdown_connections 2023-11-28T11:18:44.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.508+0000 7f97359ff700 1 -- 172.21.15.53:0/104526256 wait complete. 2023-11-28T11:18:44.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.510+0000 7f97359ff700 1 Processor -- start 2023-11-28T11:18:44.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.510+0000 7f97359ff700 1 -- start start 2023-11-28T11:18:44.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f973015c9b0 0x7f9730154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 0x7f973014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301554a0 con 0x7f97300d2320 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301529b0 con 0x7f9730161340 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f97359ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97301523f0 con 0x7f973015c9b0 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46530/0 (socket says 172.21.15.53:46530) 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 -- 172.21.15.53:0/3263855893 learned_addr learned my addr 172.21.15.53:0/3263855893 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:44.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 -- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f973015c9b0 msgr2=0x7f9730154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f973015c9b0 0x7f9730154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 -- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 msgr2=0x7f973014cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972f7fe700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 0x7f973014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 0x7f973014cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972effd700 1 -- 172.21.15.53:0/3263855893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720009550 con 0x7f97300d2320 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.512+0000 7f972f7fe700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 0x7f973014cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f972effd700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720004480 con 0x7f97300d2320 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972001f070 con 0x7f97300d2320 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97200038c0 con 0x7f97300d2320 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97300b5130 con 0x7f97300d2320 2023-11-28T11:18:44.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.513+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97300d15e0 con 0x7f97300d2320 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.514+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97300b52c0 con 0x7f97300d2320 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.516+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f972002a020 con 0x7f97300d2320 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.517+0000 7f972cff9700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 0x7f971006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.517+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f97200874b0 con 0x7f97300d2320 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.517+0000 7f9727fff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 0x7f971006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.518+0000 7f9727fff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 0x7f971006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97301543e0 tx=0x7f9718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:44.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.521+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9720053be0 con 0x7f97300d2320 2023-11-28T11:18:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.770+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97300745c0 con 0x7f97300d2320 2023-11-28T11:18:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.771+0000 7f972cff9700 1 -- 172.21.15.53:0/3263855893 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7f9720057200 con 0x7f97300d2320 2023-11-28T11:18:44.772 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:44.772 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:18:44.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 msgr2=0x7f971006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 0x7f971006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97301543e0 tx=0x7f9718008040 comp rx=0 tx=0).stop 2023-11-28T11:18:44.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 msgr2=0x7f9730151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720004750 comp rx=0 tx=0).stop 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 shutdown_connections 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f97100682f0 0x7f971006a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9730161340 0x7f973014cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97300d2320 0x7f9730151cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 --2- 172.21.15.53:0/3263855893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f973015c9b0 0x7f9730154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 >> 172.21.15.53:0/3263855893 conn(0x7f9730072600 msgr2=0x7f973015cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 shutdown_connections 2023-11-28T11:18:44.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.773+0000 7f97359ff700 1 -- 172.21.15.53:0/3263855893 wait complete. 2023-11-28T11:18:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:18:44.793 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:18:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.995+0000 7fd7195ff700 1 Processor -- start 2023-11-28T11:18:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.995+0000 7fd7195ff700 1 -- start start 2023-11-28T11:18:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.996+0000 7fd7195ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd714161340 0x7fd7140d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.996+0000 7fd7195ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.996+0000 7fd7195ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7140d1950 con 0x7fd71415c9b0 2023-11-28T11:18:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.996+0000 7fd7195ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7140b5800 con 0x7fd714161340 2023-11-28T11:18:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.996+0000 7fd7195ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140b5300 con 0x7fd7140d2320 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd712ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd71415c9b0 0x7fd7140b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35284/0 (socket says 172.21.15.53:35284) 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd714161340 0x7fd7140d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd712ffd700 1 -- 172.21.15.53:0/919996736 learned_addr learned my addr 172.21.15.53:0/919996736 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7137fe700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 -- 172.21.15.53:0/919996736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 msgr2=0x7fd7140cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 -- 172.21.15.53:0/919996736 >> v1:172.21.15.53:6789/0 conn(0x7fd71415c9b0 legacy=0x7fd7140b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 -- 172.21.15.53:0/919996736 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140b5520 con 0x7fd714161340 2023-11-28T11:18:45.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd711ffb700 1 -- 172.21.15.53:0/919996736 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576847181 0 0) 0x7fd7140d1950 con 0x7fd71415c9b0 2023-11-28T11:18:45.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.997+0000 7fd7127fc700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140d1de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd708009840 tx=0x7fd708009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cf69a29b060ce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd711ffb700 1 -- 172.21.15.53:0/919996736 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011830 con 0x7fd714161340 2023-11-28T11:18:45.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd711ffb700 1 -- 172.21.15.53:0/919996736 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd708011990 con 0x7fd714161340 2023-11-28T11:18:45.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd711ffb700 1 -- 172.21.15.53:0/919996736 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011d20 con 0x7fd714161340 2023-11-28T11:18:45.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 -- 172.21.15.53:0/919996736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 msgr2=0x7fd7140d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140d1de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd708009840 tx=0x7fd708009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 -- 172.21.15.53:0/919996736 shutdown_connections 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 --2- 172.21.15.53:0/919996736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140d1de0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 -- 172.21.15.53:0/919996736 >> 172.21.15.53:0/919996736 conn(0x7fd714072600 msgr2=0x7fd7140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 -- 172.21.15.53:0/919996736 shutdown_connections 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:44.998+0000 7fd7195ff700 1 -- 172.21.15.53:0/919996736 wait complete. 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.000+0000 7fd7195ff700 1 Processor -- start 2023-11-28T11:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.000+0000 7fd7195ff700 1 -- start start 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd71415c9b0 0x7fd71414d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140cc730 con 0x7fd714161340 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71414da60 con 0x7fd71415c9b0 2023-11-28T11:18:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7195ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140cb150 con 0x7fd7140d2320 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46556/0 (socket says 172.21.15.53:46556) 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.001+0000 7fd7137fe700 1 -- 172.21.15.53:0/742239750 learned_addr learned my addr 172.21.15.53:0/742239750 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7127fc700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd71415c9b0 0x7fd71414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd712ffd700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 -- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 msgr2=0x7fd7140cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 -- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd71415c9b0 msgr2=0x7fd71414d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd71415c9b0 0x7fd71414d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 -- 172.21.15.53:0/742239750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080094a0 con 0x7fd714161340 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd712ffd700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7137fe700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c009b20 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00f040 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cd40 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.002+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7140d15e0 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.003+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7140d1300 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.004+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71415cd80 con 0x7fd714161340 2023-11-28T11:18:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.005+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd70c014480 con 0x7fd714161340 2023-11-28T11:18:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.006+0000 7fd6fb7fe700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 0x7fd70006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.006+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd70c0869c0 con 0x7fd714161340 2023-11-28T11:18:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.006+0000 7fd712ffd700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 0x7fd70006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.007+0000 7fd712ffd700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 0x7fd70006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd704000f60 tx=0x7fd704007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.010+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd70c053090 con 0x7fd714161340 2023-11-28T11:18:45.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.263+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7140745c0 con 0x7fd714161340 2023-11-28T11:18:45.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.263+0000 7fd6fb7fe700 1 -- 172.21.15.53:0/742239750 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4652 (secure 0 0 0) 0x7fd70c018090 con 0x7fd714161340 2023-11-28T11:18:45.265 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:45.265 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4042410693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4042410693},{"type":"v1","addr":"172.21.15.157:6836","nonce":4042410693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2366197618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2366197618},{"type":"v1","addr":"172.21.15.157:6837","nonce":2366197618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3744655257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3744655257},{"type":"v1","addr":"172.21.15.53:6834","nonce":3744655257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:17:49.188727+0000","modified":"2023-11-28T11:17:52.679701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2501355604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2501355604},{"type":"v1","addr":"172.21.15.53:6835","nonce":2501355604}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 msgr2=0x7fd70006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 0x7fd70006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd704000f60 tx=0x7fd704007d30 comp rx=0 tx=0).stop 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 msgr2=0x7fd7140c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c009f60 comp rx=0 tx=0).stop 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 shutdown_connections 2023-11-28T11:18:45.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7000682f0 0x7fd70006a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7140d2320 0x7fd7140cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd714161340 0x7fd7140c9160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 --2- 172.21.15.53:0/742239750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd71415c9b0 0x7fd71414d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 >> 172.21.15.53:0/742239750 conn(0x7fd714072600 msgr2=0x7fd71415f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 shutdown_connections 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.266+0000 7fd7195ff700 1 -- 172.21.15.53:0/742239750 wait complete. 2023-11-28T11:18:45.268 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-28T11:18:45.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:45.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.485+0000 7f7db93ff700 1 Processor -- start 2023-11-28T11:18:45.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.485+0000 7f7db93ff700 1 -- start start 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db93ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7db415c9b0 0x7f7db40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db93ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7db40cd7c0 0x7f7db40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db93ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db40d1950 con 0x7f7db4161340 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db93ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40b5800 con 0x7f7db415c9b0 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db93ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40ce140 con 0x7f7db40cd7c0 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.486+0000 7f7db2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7db415c9b0 0x7f7db40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7db415c9b0 0x7f7db40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46572/0 (socket says 172.21.15.53:46572) 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3552571982 learned_addr learned my addr 172.21.15.53:0/3552571982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3552571982 >> v2:172.21.15.157:3300/0 conn(0x7f7db40cd7c0 msgr2=0x7f7db40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:45.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 --2- 172.21.15.53:0/3552571982 >> v2:172.21.15.157:3300/0 conn(0x7f7db40cd7c0 0x7f7db40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3552571982 >> v1:172.21.15.157:6789/0 conn(0x7f7db4161340 legacy=0x7f7db40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:45.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3552571982 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40d24f0 con 0x7f7db415c9b0 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.487+0000 7f7db2ffd700 1 --2- 172.21.15.53:0/3552571982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db415c9b0 0x7f7db40b4bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7db406c170 tx=0x7f7da4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcc249b13c19b73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db1ffb700 1 -- 172.21.15.53:0/3552571982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4004230 con 0x7f7db415c9b0 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db1ffb700 1 -- 172.21.15.53:0/3552571982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4004390 con 0x7f7db415c9b0 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db1ffb700 1 -- 172.21.15.53:0/3552571982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da40036a0 con 0x7f7db415c9b0 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 -- 172.21.15.53:0/3552571982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db415c9b0 msgr2=0x7f7db40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3552571982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db415c9b0 0x7f7db40b4bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7db406c170 tx=0x7f7da4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 -- 172.21.15.53:0/3552571982 shutdown_connections 2023-11-28T11:18:45.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3552571982 >> v2:172.21.15.157:3300/0 conn(0x7f7db40cd7c0 0x7f7db40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3552571982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db415c9b0 0x7f7db40b4bf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 -- 172.21.15.53:0/3552571982 >> 172.21.15.53:0/3552571982 conn(0x7f7db4072600 msgr2=0x7f7db40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 -- 172.21.15.53:0/3552571982 shutdown_connections 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.488+0000 7f7db93ff700 1 -- 172.21.15.53:0/3552571982 wait complete. 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.490+0000 7f7db93ff700 1 Processor -- start 2023-11-28T11:18:45.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.490+0000 7f7db93ff700 1 -- start start 2023-11-28T11:18:45.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db415c9b0 0x7f7db4158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 0x7f7db4158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4155770 con 0x7f7db40cd7c0 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40c7b90 con 0x7f7db415c9b0 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4154dd0 con 0x7f7db4161340 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46578/0 (socket says 172.21.15.53:46578) 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3894121912 learned_addr learned my addr 172.21.15.53:0/3894121912 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.491+0000 7f7db27fc700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db415c9b0 0x7f7db4158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 msgr2=0x7f7db4158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db37fe700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 0x7f7db4158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 0x7f7db4158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db415c9b0 msgr2=0x7f7db4158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db415c9b0 0x7f7db4158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 -- 172.21.15.53:0/3894121912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da4009550 con 0x7f7db40cd7c0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db37fe700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 0x7f7db4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db2ffd700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7db40675d0 tx=0x7f7da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011630 con 0x7f7db40cd7c0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4011790 con 0x7f7db40cd7c0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011b40 con 0x7f7db40cd7c0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db40d1300 con 0x7f7db40cd7c0 2023-11-28T11:18:45.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.492+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db40b5520 con 0x7f7db40cd7c0 2023-11-28T11:18:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.493+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db40d15e0 con 0x7f7db40cd7c0 2023-11-28T11:18:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.494+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7da4017020 con 0x7f7db40cd7c0 2023-11-28T11:18:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.495+0000 7f7d9b7fe700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 0x7f7da006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.495+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7da4087400 con 0x7f7db40cd7c0 2023-11-28T11:18:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.496+0000 7f7db27fc700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 0x7f7da006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.496+0000 7f7db27fc700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 0x7f7da006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db40b70e0 tx=0x7f7da8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.499+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7da4053b30 con 0x7f7db40cd7c0 2023-11-28T11:18:45.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.703+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db415fd30 con 0x7f7db40cd7c0 2023-11-28T11:18:45.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.705+0000 7f7d9b7fe700 1 -- 172.21.15.53:0/3894121912 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18002 (secure 0 0 0) 0x7f7da4057150 con 0x7f7db40cd7c0 2023-11-28T11:18:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:45.706 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:36.870456+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:17:47.656143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:17:48.158158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.9","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 msgr2=0x7f7da006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 0x7f7da006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db40b70e0 tx=0x7f7da8008040 comp rx=0 tx=0).stop 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 msgr2=0x7f7db414a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7db40675d0 tx=0x7f7da4004750 comp rx=0 tx=0).stop 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 shutdown_connections 2023-11-28T11:18:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7da00682f0 0x7f7da006a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db4161340 0x7f7db4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7db40cd7c0 0x7f7db414a580 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 --2- 172.21.15.53:0/3894121912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db415c9b0 0x7f7db4158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 >> 172.21.15.53:0/3894121912 conn(0x7f7db4072600 msgr2=0x7f7db40b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 shutdown_connections 2023-11-28T11:18:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.707+0000 7f7db93ff700 1 -- 172.21.15.53:0/3894121912 wait complete. 2023-11-28T11:18:45.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:18:45.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.930+0000 7fe04a7ff700 1 Processor -- start 2023-11-28T11:18:45.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.930+0000 7fe04a7ff700 1 -- start start 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe04a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe04406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe04a7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440cf5e0 con 0x7fe0440cce10 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe04a7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440cecb0 con 0x7fe04406b8c0 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe04a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440cd290 con 0x7fe04415ca50 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe048d9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe0440cce10 0x7fe0440cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35286/0 (socket says 172.21.15.53:35286) 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe048d9c700 1 -- 172.21.15.53:0/3506628992 learned_addr learned my addr 172.21.15.53:0/3506628992 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:45.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe043fff700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe04406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739796220 0 0) 0x7fe0440cf5e0 con 0x7fe0440cce10 2023-11-28T11:18:45.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.932+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe02c0035b0 con 0x7fe0440cce10 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233192906 0 0) 0x7fe0440cecb0 con 0x7fe04406b8c0 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0440cf5e0 con 0x7fe04406b8c0 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413586712 0 0) 0x7fe02c0035b0 con 0x7fe0440cce10 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 msgr2=0x7fe04406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe04406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 >> v1:172.21.15.157:6789/0 conn(0x7fe04406b8c0 legacy=0x7fe0440cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0440cef90 con 0x7fe0440cce10 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fe03c0019f0 con 0x7fe0440cce10 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 >> v1:172.21.15.53:6789/0 conn(0x7fe0440cce10 legacy=0x7fe0440cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02c003bc0 0x7fe02c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe02c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe02c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440cd290 con 0x7fe02c004d90 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c00bac0 con 0x7fe02c003bc0 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c00bc30 con 0x7fe04415ca50 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe048d9c700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe02c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe043fff700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02c003bc0 0x7fe02c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.933+0000 7fe0437fe700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe02c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 -- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 msgr2=0x7fe02c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:45.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe04415ca50 0x7fe02c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 -- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02c003bc0 msgr2=0x7fe02c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe02c003bc0 0x7fe02c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 -- 172.21.15.53:0/3506628992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0035b0 con 0x7fe02c004d90 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe02c005200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00a7a0 tx=0x7fe02c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda85d51da6ff75 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0072d0 con 0x7fe02c004d90 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c007450 con 0x7fe02c004d90 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe042ffd700 1 -- 172.21.15.53:0/3506628992 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c007800 con 0x7fe02c004d90 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe04a7ff700 1 -- 172.21.15.53:0/3506628992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 msgr2=0x7fe02c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/3506628992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe02c005200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00a7a0 tx=0x7fe02c005880 comp rx=0 tx=0).stop 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.934+0000 7fe048d9c700 1 -- 172.21.15.53:0/3506628992 reap_dead start 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.935+0000 7fe04a7ff700 1 -- 172.21.15.53:0/3506628992 shutdown_connections 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.935+0000 7fe04a7ff700 1 -- 172.21.15.53:0/3506628992 >> 172.21.15.53:0/3506628992 conn(0x7fe04409a230 msgr2=0x7fe04408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:45.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.935+0000 7fe04a7ff700 1 -- 172.21.15.53:0/3506628992 shutdown_connections 2023-11-28T11:18:45.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.935+0000 7fe04a7ff700 1 -- 172.21.15.53:0/3506628992 wait complete. 2023-11-28T11:18:45.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.936+0000 7fe04a7ff700 1 Processor -- start 2023-11-28T11:18:45.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 -- start start 2023-11-28T11:18:45.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0440c57e0 0x7fe0440c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0440c6660 0x7fe044157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04414a940 con 0x7fe02c004d90 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c5350 con 0x7fe0440c57e0 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.937+0000 7fe04a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c4a60 con 0x7fe0440c6660 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46596/0 (socket says 172.21.15.53:46596) 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 -- 172.21.15.53:0/1637566614 learned_addr learned my addr 172.21.15.53:0/1637566614 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0440c57e0 0x7fe0440c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe048d9c700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0440c6660 0x7fe044157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 -- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0440c6660 msgr2=0x7fe044157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0440c6660 0x7fe044157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 -- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0440c57e0 msgr2=0x7fe0440c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0440c57e0 0x7fe0440c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 -- 172.21.15.53:0/1637566614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04415ce20 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe043fff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe034011aa0 tx=0x7fe034011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.938+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400f980 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.939+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400fae0 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.939+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400fe90 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.939+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0440cf270 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.939+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04415cfb0 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.940+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0440c5630 con 0x7fe02c004d90 2023-11-28T11:18:45.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.940+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe034014440 con 0x7fe02c004d90 2023-11-28T11:18:45.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.942+0000 7fe040ff9700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 0x7fe03006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:45.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.942+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe034086770 con 0x7fe02c004d90 2023-11-28T11:18:45.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.942+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 0x7fe03006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:45.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.943+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 0x7fe03006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe0440aefb0 tx=0x7fe038009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:45.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:45.946+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe034052f00 con 0x7fe02c004d90 2023-11-28T11:18:46.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.195+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe0440745c0 con 0x7fe02c004d90 2023-11-28T11:18:46.909 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-11-28T11:18:46.908+0000 7fa6ddbf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:18:46.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.915+0000 7fe040ff9700 1 -- 172.21.15.53:0/1637566614 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe034056520 con 0x7fe02c004d90 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 msgr2=0x7fe03006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 0x7fe03006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe0440aefb0 tx=0x7fe038009040 comp rx=0 tx=0).stop 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 msgr2=0x7fe04414a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe034011aa0 tx=0x7fe034011e60 comp rx=0 tx=0).stop 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 shutdown_connections 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe0300682f0 0x7fe03006a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0440c6660 0x7fe044157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:46.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c004d90 0x7fe04414a400 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 --2- 172.21.15.53:0/1637566614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0440c57e0 0x7fe0440c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 >> 172.21.15.53:0/1637566614 conn(0x7fe04409a230 msgr2=0x7fe0440cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 shutdown_connections 2023-11-28T11:18:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:46.919+0000 7fe04a7ff700 1 -- 172.21.15.53:0/1637566614 wait complete. 2023-11-28T11:18:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:18:46.938 INFO:tasks.ceph.mds.c.smithi053.stderr: -7173> 2023-11-28T11:17:54.659+0000 7f6a025f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:18:46.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:18:46.998 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:18:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.160+0000 7fb24e7ff700 1 Processor -- start 2023-11-28T11:18:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.160+0000 7fb24e7ff700 1 -- start start 2023-11-28T11:18:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24e7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24e7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb248161340 0x7fb2480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24e7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2480d1950 con 0x7fb2480d2320 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24e7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480b5800 con 0x7fb24815c9b0 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24e7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b5520 con 0x7fb248161340 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46618/0 (socket says 172.21.15.53:46618) 2023-11-28T11:18:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb24cd9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb2480d2320 0x7fb2480cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35298/0 (socket says 172.21.15.53:35298) 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 -- 172.21.15.53:0/3135909394 learned_addr learned my addr 172.21.15.53:0/3135909394 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb2477fe700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb248161340 0x7fb2480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 -- 172.21.15.53:0/3135909394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb248161340 msgr2=0x7fb2480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb248161340 0x7fb2480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 -- 172.21.15.53:0/3135909394 >> v1:172.21.15.53:6789/0 conn(0x7fb2480d2320 legacy=0x7fb2480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb247fff700 1 -- 172.21.15.53:0/3135909394 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480d1300 con 0x7fb24815c9b0 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.162+0000 7fb246ffd700 1 -- 172.21.15.53:0/3135909394 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289195926 0 0) 0x7fb2480d1950 con 0x7fb2480d2320 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb247fff700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2380098f0 tx=0x7fb238009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a14877a4b449e61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb246ffd700 1 -- 172.21.15.53:0/3135909394 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800b8a0 con 0x7fb24815c9b0 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb246ffd700 1 -- 172.21.15.53:0/3135909394 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23800ba00 con 0x7fb24815c9b0 2023-11-28T11:18:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb246ffd700 1 -- 172.21.15.53:0/3135909394 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800bd90 con 0x7fb24815c9b0 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3135909394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 msgr2=0x7fb2480b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.163+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2380098f0 tx=0x7fb238009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3135909394 shutdown_connections 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3135909394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb248161340 0x7fb2480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3135909394 >> 172.21.15.53:0/3135909394 conn(0x7fb248072600 msgr2=0x7fb2480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3135909394 shutdown_connections 2023-11-28T11:18:47.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.164+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3135909394 wait complete. 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.165+0000 7fb24e7ff700 1 Processor -- start 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 -- start start 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480d2320 0x7fb2480c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb24815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2481555f0 con 0x7fb24815c9b0 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480c6ce0 con 0x7fb248161340 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.166+0000 7fb24e7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480c9fe0 con 0x7fb2480d2320 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46626/0 (socket says 172.21.15.53:46626) 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 -- 172.21.15.53:0/3208704501 learned_addr learned my addr 172.21.15.53:0/3208704501 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb247fff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480d2320 0x7fb2480c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb24cd9c700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb24815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 -- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480d2320 msgr2=0x7fb2480c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480d2320 0x7fb2480c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 -- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 msgr2=0x7fb24815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb24815f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 -- 172.21.15.53:0/3208704501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb238009550 con 0x7fb24815c9b0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.167+0000 7fb2477fe700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e810 tx=0x7fb23c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.168+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c013610 con 0x7fb24815c9b0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.168+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c013770 con 0x7fb24815c9b0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.168+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480b5130 con 0x7fb24815c9b0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.168+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c013b20 con 0x7fb24815c9b0 2023-11-28T11:18:47.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.168+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2480d15e0 con 0x7fb24815c9b0 2023-11-28T11:18:47.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.169+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2480b52c0 con 0x7fb24815c9b0 2023-11-28T11:18:47.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.170+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb23c023070 con 0x7fb24815c9b0 2023-11-28T11:18:47.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.171+0000 7fb244ff9700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 0x7fb23406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:47.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.171+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fb23c085b30 con 0x7fb24815c9b0 2023-11-28T11:18:47.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.171+0000 7fb247fff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 0x7fb23406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:47.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.172+0000 7fb247fff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 0x7fb23406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c40 tx=0x7fb238008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:47.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.175+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb23c052190 con 0x7fb24815c9b0 2023-11-28T11:18:47.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.421+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2480d2750 con 0x7fb24815c9b0 2023-11-28T11:18:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.919+0000 7fb244ff9700 1 -- 172.21.15.53:0/3208704501 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb23c0557b0 con 0x7fb24815c9b0 2023-11-28T11:18:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 msgr2=0x7fb23406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 0x7fb23406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb238009c40 tx=0x7fb238008030 comp rx=0 tx=0).stop 2023-11-28T11:18:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 msgr2=0x7fb248154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e810 tx=0x7fb23c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 shutdown_connections 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2340682f0 0x7fb23406a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480d2320 0x7fb2480c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb24815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 --2- 172.21.15.53:0/3208704501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb248154180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 >> 172.21.15.53:0/3208704501 conn(0x7fb248072600 msgr2=0x7fb24815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 shutdown_connections 2023-11-28T11:18:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:47.922+0000 7fb24e7ff700 1 -- 172.21.15.53:0/3208704501 wait complete. 2023-11-28T11:18:47.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:18:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.153+0000 7fce987ff700 1 Processor -- start 2023-11-28T11:18:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.153+0000 7fce987ff700 1 -- start start 2023-11-28T11:18:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce987ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce900cded0 0x7fce90161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce987ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce9015dc30 0x7fce900b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce987ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce90160d70 con 0x7fce9015a760 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce987ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9015e0d0 con 0x7fce900cded0 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce987ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9015ab30 con 0x7fce9015dc30 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce900cded0 0x7fce90161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce900cded0 0x7fce90161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46638/0 (socket says 172.21.15.53:46638) 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 -- 172.21.15.53:0/1070498626 learned_addr learned my addr 172.21.15.53:0/1070498626 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce96d9c700 1 --2- 172.21.15.53:0/1070498626 >> v2:172.21.15.157:3300/0 conn(0x7fce9015dc30 0x7fce900b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 -- 172.21.15.53:0/1070498626 >> v2:172.21.15.157:3300/0 conn(0x7fce9015dc30 msgr2=0x7fce900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 --2- 172.21.15.53:0/1070498626 >> v2:172.21.15.157:3300/0 conn(0x7fce9015dc30 0x7fce900b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 -- 172.21.15.53:0/1070498626 >> v1:172.21.15.157:6789/0 conn(0x7fce9015a760 legacy=0x7fce900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.155+0000 7fce95d9a700 1 -- 172.21.15.53:0/1070498626 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce900b5130 con 0x7fce900cded0 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce95d9a700 1 --2- 172.21.15.53:0/1070498626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce900cded0 0x7fce90161200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce900674d0 tx=0x7fce80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c6768ed20cea6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce95599700 1 -- 172.21.15.53:0/1070498626 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011830 con 0x7fce900cded0 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce95599700 1 -- 172.21.15.53:0/1070498626 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80011990 con 0x7fce900cded0 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce95599700 1 -- 172.21.15.53:0/1070498626 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011d20 con 0x7fce900cded0 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce987ff700 1 -- 172.21.15.53:0/1070498626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce900cded0 msgr2=0x7fce90161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.156+0000 7fce987ff700 1 --2- 172.21.15.53:0/1070498626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce900cded0 0x7fce90161200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce900674d0 tx=0x7fce80009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 -- 172.21.15.53:0/1070498626 shutdown_connections 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 --2- 172.21.15.53:0/1070498626 >> v2:172.21.15.157:3300/0 conn(0x7fce9015dc30 0x7fce900b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 --2- 172.21.15.53:0/1070498626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce900cded0 0x7fce90161200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 -- 172.21.15.53:0/1070498626 >> 172.21.15.53:0/1070498626 conn(0x7fce90072620 msgr2=0x7fce900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 -- 172.21.15.53:0/1070498626 shutdown_connections 2023-11-28T11:18:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.157+0000 7fce987ff700 1 -- 172.21.15.53:0/1070498626 wait complete. 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.158+0000 7fce987ff700 1 Processor -- start 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 -- start start 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce900cded0 0x7fce900c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce9015a760 0x7fce900cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900ca240 con 0x7fce9015dc30 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900c9080 con 0x7fce9015a760 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.159+0000 7fce987ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce900c9360 con 0x7fce900cded0 2023-11-28T11:18:48.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46654/0 (socket says 172.21.15.53:46654) 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 -- 172.21.15.53:0/1448345533 learned_addr learned my addr 172.21.15.53:0/1448345533 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce9659b700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce900cded0 0x7fce900c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce95d9a700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce9015a760 0x7fce900cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 -- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce900cded0 msgr2=0x7fce900c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce900cded0 0x7fce900c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 -- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce9015a760 msgr2=0x7fce900cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce9015a760 0x7fce900cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.160+0000 7fce96d9c700 1 -- 172.21.15.53:0/1448345533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80009550 con 0x7fce9015dc30 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce96d9c700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fce8800efc0 tx=0x7fce8800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800cad0 con 0x7fce9015dc30 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800f040 con 0x7fce9015dc30 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce900b5670 con 0x7fce9015dc30 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800ce70 con 0x7fce9015dc30 2023-11-28T11:18:48.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.161+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce900b5390 con 0x7fce9015dc30 2023-11-28T11:18:48.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.162+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90160720 con 0x7fce9015dc30 2023-11-28T11:18:48.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.163+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce88014480 con 0x7fce9015dc30 2023-11-28T11:18:48.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.164+0000 7fce86ffd700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 0x7fce7406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:48.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.164+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fce88086720 con 0x7fce9015dc30 2023-11-28T11:18:48.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.164+0000 7fce9659b700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 0x7fce7406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:48.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.165+0000 7fce9659b700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 0x7fce7406a7d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007fd0 tx=0x7fce8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:48.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.168+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce88052d10 con 0x7fce9015dc30 2023-11-28T11:18:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.370+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce900c9a40 con 0x7fce9015dc30 2023-11-28T11:18:48.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.920+0000 7fce86ffd700 1 -- 172.21.15.53:0/1448345533 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce880180d0 con 0x7fce9015dc30 2023-11-28T11:18:48.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.924+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 msgr2=0x7fce7406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.924+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 0x7fce7406a7d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007fd0 tx=0x7fce8c009040 comp rx=0 tx=0).stop 2023-11-28T11:18:48.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.924+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 msgr2=0x7fce900d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:48.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.924+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fce8800efc0 tx=0x7fce8800c0c0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 shutdown_connections 2023-11-28T11:18:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fce74068320 0x7fce7406a7d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce900cded0 0x7fce900c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce9015dc30 0x7fce900d12d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 --2- 172.21.15.53:0/1448345533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce9015a760 0x7fce900cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.925+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 >> 172.21.15.53:0/1448345533 conn(0x7fce90072620 msgr2=0x7fce900cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.926+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 shutdown_connections 2023-11-28T11:18:48.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:48.926+0000 7fce987ff700 1 -- 172.21.15.53:0/1448345533 wait complete. 2023-11-28T11:18:48.928 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:18:48.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:18:49.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.155+0000 7f9629fff700 1 Processor -- start 2023-11-28T11:18:49.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.156+0000 7f9629fff700 1 -- start start 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f9629fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f96240cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f9629fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9624157530 con 0x7f96240af8e0 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f9629fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96240c9820 con 0x7f9624157160 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f9629fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96240c9b00 con 0x7f96240afd60 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f961bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f96240af8e0 0x7f96240caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35310/0 (socket says 172.21.15.53:35310) 2023-11-28T11:18:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f961bfff700 1 -- 172.21.15.53:0/3445174097 learned_addr learned my addr 172.21.15.53:0/3445174097 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:49.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.157+0000 7f9623fff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f96240cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691103932 0 0) 0x7f9624157530 con 0x7f96240af8e0 2023-11-28T11:18:49.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96100035b0 con 0x7f96240af8e0 2023-11-28T11:18:49.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600688763 0 0) 0x7f96240c9820 con 0x7f9624157160 2023-11-28T11:18:49.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9624157530 con 0x7f9624157160 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309280290 0 0) 0x7f96100035b0 con 0x7f96240af8e0 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 msgr2=0x7f96240cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f96240cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 >> v1:172.21.15.157:6789/0 conn(0x7f9624157160 legacy=0x7f962415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240c9de0 con 0x7f96240af8e0 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f960c0017f0 con 0x7f96240af8e0 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 >> v1:172.21.15.53:6789/0 conn(0x7f96240af8e0 legacy=0x7f96240caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f9610003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9610004120 0x7f9610004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004d90 0x7f9610005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240c9b00 con 0x7f9610004120 2023-11-28T11:18:49.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961000ddb0 con 0x7f9610004d90 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961000df40 con 0x7f96240afd60 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.158+0000 7f961bfff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9610004120 0x7f9610004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f96237fe700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f9610003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f9623fff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004d90 0x7f9610005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 -- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 msgr2=0x7f9610003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96240afd60 0x7f9610003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 -- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004d90 msgr2=0x7f9610005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004d90 0x7f9610005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 -- 172.21.15.53:0/3445174097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96100035b0 con 0x7f9610004120 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f961bfff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9610004120 0x7f9610004850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c240 tx=0x7f9610005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073e115adb0c915 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00cd40 con 0x7f9610004120 2023-11-28T11:18:49.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c007b50 con 0x7f9610004120 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.159+0000 7f9622ffd700 1 -- 172.21.15.53:0/3445174097 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c007310 con 0x7f9610004120 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 -- 172.21.15.53:0/3445174097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9610004120 msgr2=0x7f9610004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 --2- 172.21.15.53:0/3445174097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9610004120 0x7f9610004850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c240 tx=0x7f9610005880 comp rx=0 tx=0).stop 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9623fff700 1 -- 172.21.15.53:0/3445174097 reap_dead start 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 -- 172.21.15.53:0/3445174097 shutdown_connections 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 -- 172.21.15.53:0/3445174097 >> 172.21.15.53:0/3445174097 conn(0x7f96240b0860 msgr2=0x7f96240b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 -- 172.21.15.53:0/3445174097 shutdown_connections 2023-11-28T11:18:49.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.160+0000 7f9629fff700 1 -- 172.21.15.53:0/3445174097 wait complete. 2023-11-28T11:18:49.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.162+0000 7f9629fff700 1 Processor -- start 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.162+0000 7f9629fff700 1 -- start start 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 0x7f96241497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96241e4910 0x7f96240adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240ad2c0 con 0x7f96241e25b0 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96241e2f70 con 0x7f9610004120 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9629fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9624149cf0 con 0x7f96241e4910 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9623fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f96237fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 0x7f96241497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9623fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46664/0 (socket says 172.21.15.53:46664) 2023-11-28T11:18:49.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f9623fff700 1 -- 172.21.15.53:0/2154340877 learned_addr learned my addr 172.21.15.53:0/2154340877 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.163+0000 7f961bfff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96241e4910 0x7f96240adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 -- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96241e4910 msgr2=0x7f96240adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96241e4910 0x7f96240adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 -- 172.21.15.53:0/2154340877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 msgr2=0x7f96241e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 -- 172.21.15.53:0/2154340877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240ca3a0 con 0x7f9610004120 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f9623fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f96237fe700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 0x7f96241497b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f961400f770 tx=0x7f961400fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.164+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614005320 con 0x7f9610004120 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.165+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96240ca0c0 con 0x7f9610004120 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.165+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140054a0 con 0x7f9610004120 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.165+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614006aa0 con 0x7f9610004120 2023-11-28T11:18:49.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.165+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96240b0180 con 0x7f9610004120 2023-11-28T11:18:49.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.166+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96240ce690 con 0x7f9610004120 2023-11-28T11:18:49.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.167+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9614006c00 con 0x7f9610004120 2023-11-28T11:18:49.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.168+0000 7f9620ff9700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 0x7f960406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:49.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.168+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f96140858f0 con 0x7f9610004120 2023-11-28T11:18:49.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.168+0000 7f961bfff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 0x7f960406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:49.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.169+0000 7f961bfff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 0x7f960406a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c240 tx=0x7f960c00a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:49.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.172+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9614052180 con 0x7f9610004120 2023-11-28T11:18:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.377+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f962404e620 con 0x7f9610004120 2023-11-28T11:18:49.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.930+0000 7f9620ff9700 1 -- 172.21.15.53:0/2154340877 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f96140557a0 con 0x7f9610004120 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 msgr2=0x7f960406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 0x7f960406a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c240 tx=0x7f960c00a000 comp rx=0 tx=0).stop 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 msgr2=0x7f96241497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 0x7f96241497b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f961400f770 tx=0x7f961400fa80 comp rx=0 tx=0).stop 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 shutdown_connections 2023-11-28T11:18:49.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f96040682f0 0x7f960406a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96241e25b0 0x7f96241e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9610004120 0x7f96241497b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 --2- 172.21.15.53:0/2154340877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96241e4910 0x7f96240adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 >> 172.21.15.53:0/2154340877 conn(0x7f96240b0860 msgr2=0x7f96240c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 shutdown_connections 2023-11-28T11:18:49.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:49.936+0000 7f9629fff700 1 -- 172.21.15.53:0/2154340877 wait complete. 2023-11-28T11:18:49.939 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-11-28T11:18:49.962 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:18:49.962 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:18:49.962 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:18:49.962 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:18:49.963 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:18:49.963 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:18:49.963 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:18:49.963 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:18:49.963 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:18:49.963 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:18:49.964 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:18:49.964 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:18:49.964 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:18:49.964 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:18:49.964 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:18:49.964 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:18:49.978 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:18:49.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:18:49.979 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:18:49.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:18:49.981 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:18:49.981 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:18:49.982 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:18:49.982 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:18:50.022 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:18:50.023 DEBUG:teuthology.parallel:result is None 2023-11-28T11:18:50.025 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:18:50.025 DEBUG:teuthology.parallel:result is None 2023-11-28T11:18:50.026 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:18:50.026 DEBUG:teuthology.parallel:result is None 2023-11-28T11:18:50.028 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:18:50.028 DEBUG:teuthology.parallel:result is None 2023-11-28T11:18:50.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:18:50.175 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-11-28T11:18:50.176 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-11-28T11:18:50.176 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:18:50.177 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:18:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.255+0000 7f45d77ff700 1 Processor -- start 2023-11-28T11:18:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.255+0000 7f45d77ff700 1 -- start start 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45d015c9b0 0x7f45d00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45d00d2320 0x7f45d00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00b5800 con 0x7f45d00d2320 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00d1950 con 0x7f45d015c9b0 2023-11-28T11:18:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.256+0000 7f45d77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00b5520 con 0x7f45d0161340 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45d00d2320 0x7f45d00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45d00d2320 0x7f45d00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46708/0 (socket says 172.21.15.53:46708) 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 -- 172.21.15.53:0/4182323850 learned_addr learned my addr 172.21.15.53:0/4182323850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d559b700 1 --2- 172.21.15.53:0/4182323850 >> v2:172.21.15.157:3300/0 conn(0x7f45d015c9b0 0x7f45d00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d4d9a700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 -- 172.21.15.53:0/4182323850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 msgr2=0x7f45d00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 -- 172.21.15.53:0/4182323850 >> v2:172.21.15.157:3300/0 conn(0x7f45d015c9b0 msgr2=0x7f45d00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 --2- 172.21.15.53:0/4182323850 >> v2:172.21.15.157:3300/0 conn(0x7f45d015c9b0 0x7f45d00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 -- 172.21.15.53:0/4182323850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00d1300 con 0x7f45d00d2320 2023-11-28T11:18:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.257+0000 7f45d5d9c700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00cd7c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45d0067460 tx=0x7f45c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15aa6210736f2225 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45c7fff700 1 -- 172.21.15.53:0/4182323850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800e070 con 0x7f45d00d2320 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45c7fff700 1 -- 172.21.15.53:0/4182323850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800b9c0 con 0x7f45d00d2320 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45c7fff700 1 -- 172.21.15.53:0/4182323850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800bd70 con 0x7f45d00d2320 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 -- 172.21.15.53:0/4182323850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 msgr2=0x7f45d00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00cd7c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45d0067460 tx=0x7f45c800b330 comp rx=0 tx=0).stop 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 -- 172.21.15.53:0/4182323850 shutdown_connections 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00cd7c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4182323850 >> v2:172.21.15.157:3300/0 conn(0x7f45d015c9b0 0x7f45d00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4182323850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 -- 172.21.15.53:0/4182323850 >> 172.21.15.53:0/4182323850 conn(0x7f45d0072600 msgr2=0x7f45d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 -- 172.21.15.53:0/4182323850 shutdown_connections 2023-11-28T11:18:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.258+0000 7f45d77ff700 1 -- 172.21.15.53:0/4182323850 wait complete. 2023-11-28T11:18:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.260+0000 7f45d77ff700 1 Processor -- start 2023-11-28T11:18:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.260+0000 7f45d77ff700 1 -- start start 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d015c9b0 0x7f45d01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c9a00 con 0x7f45d00d2320 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c9ce0 con 0x7f45d015c9b0 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c9fc0 con 0x7f45d0161340 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d559b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d559b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46718/0 (socket says 172.21.15.53:46718) 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.261+0000 7f45d559b700 1 -- 172.21.15.53:0/4110250559 learned_addr learned my addr 172.21.15.53:0/4110250559 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 -- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 msgr2=0x7f45d00c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 -- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d015c9b0 msgr2=0x7f45d01f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d015c9b0 0x7f45d01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 -- 172.21.15.53:0/4110250559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8009b30 con 0x7f45d00d2320 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d559b700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d800 tx=0x7f45cc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0125f0 con 0x7f45d00d2320 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc012750 con 0x7f45d00d2320 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d00b5130 con 0x7f45d00d2320 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc012ae0 con 0x7f45d00d2320 2023-11-28T11:18:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.262+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d00d15e0 con 0x7f45d00d2320 2023-11-28T11:18:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.263+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d00b52c0 con 0x7f45d00d2320 2023-11-28T11:18:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.264+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45cc012c40 con 0x7f45d00d2320 2023-11-28T11:18:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.265+0000 7f45c5ffb700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 0x7f45b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.265+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f45cc085f80 con 0x7f45d00d2320 2023-11-28T11:18:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.265+0000 7f45d4d9a700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 0x7f45b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.266+0000 7f45d4d9a700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 0x7f45b406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ce3a0 tx=0x7f45c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:50.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.269+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45cc052a10 con 0x7f45d00d2320 2023-11-28T11:18:50.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:50.485+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45d00745c0 con 0x7f45d00d2320 2023-11-28T11:18:51.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.210+0000 7f45c5ffb700 1 -- 172.21.15.53:0/4110250559 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45cc012ed0 con 0x7f45d00d2320 2023-11-28T11:18:51.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 msgr2=0x7f45b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 0x7f45b406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ce3a0 tx=0x7f45c0008040 comp rx=0 tx=0).stop 2023-11-28T11:18:51.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 msgr2=0x7f45d00c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d800 tx=0x7f45cc00dbc0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 shutdown_connections 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f45b40682f0 0x7f45b406a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0161340 0x7f45d00c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d00d2320 0x7f45d00c9320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 --2- 172.21.15.53:0/4110250559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d015c9b0 0x7f45d01f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 >> 172.21.15.53:0/4110250559 conn(0x7f45d0072600 msgr2=0x7f45d006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 shutdown_connections 2023-11-28T11:18:51.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.213+0000 7f45d77ff700 1 -- 172.21.15.53:0/4110250559 wait complete. 2023-11-28T11:18:51.216 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-11-28T11:18:51.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:18:51.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.437+0000 7fcfbddff700 1 Processor -- start 2023-11-28T11:18:51.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.437+0000 7fcfbddff700 1 -- start start 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfbddff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfb8161340 0x7fcfb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfbddff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80d1950 con 0x7fcfb80d2320 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfbddff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80b5520 con 0x7fcfb815c9b0 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfbddff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb80b5800 con 0x7fcfb8161340 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb6ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfb8161340 0x7fcfb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb7fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcfb80d2320 0x7fcfb80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39988/0 (socket says 172.21.15.53:39988) 2023-11-28T11:18:51.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb7fff700 1 -- 172.21.15.53:0/2841010584 learned_addr learned my addr 172.21.15.53:0/2841010584 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:51.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb6ffd700 1 -- 172.21.15.53:0/2841010584 >> v1:172.21.15.157:6789/0 conn(0x7fcfb815c9b0 legacy=0x7fcfb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb6ffd700 1 -- 172.21.15.53:0/2841010584 >> v1:172.21.15.53:6789/0 conn(0x7fcfb80d2320 legacy=0x7fcfb80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb67fc700 1 -- 172.21.15.53:0/2841010584 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004212276 0 0) 0x7fcfb80d1950 con 0x7fcfb80d2320 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.439+0000 7fcfb6ffd700 1 -- 172.21.15.53:0/2841010584 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80d1300 con 0x7fcfb8161340 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfb6ffd700 1 --2- 172.21.15.53:0/2841010584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8161340 0x7fcfb80d1de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009840 tx=0x7fcfac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404339bb79faa991 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfb67fc700 1 -- 172.21.15.53:0/2841010584 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac011830 con 0x7fcfb8161340 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfb67fc700 1 -- 172.21.15.53:0/2841010584 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac011990 con 0x7fcfb8161340 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfb67fc700 1 -- 172.21.15.53:0/2841010584 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac011d20 con 0x7fcfb8161340 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfbddff700 1 -- 172.21.15.53:0/2841010584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8161340 msgr2=0x7fcfb80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.440+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2841010584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8161340 0x7fcfb80d1de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009840 tx=0x7fcfac009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.441+0000 7fcfbddff700 1 -- 172.21.15.53:0/2841010584 shutdown_connections 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.441+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2841010584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8161340 0x7fcfb80d1de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.441+0000 7fcfbddff700 1 -- 172.21.15.53:0/2841010584 >> 172.21.15.53:0/2841010584 conn(0x7fcfb8072600 msgr2=0x7fcfb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:51.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.441+0000 7fcfbddff700 1 -- 172.21.15.53:0/2841010584 shutdown_connections 2023-11-28T11:18:51.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.441+0000 7fcfbddff700 1 -- 172.21.15.53:0/2841010584 wait complete. 2023-11-28T11:18:51.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.442+0000 7fcfbddff700 1 Processor -- start 2023-11-28T11:18:51.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.442+0000 7fcfbddff700 1 -- start start 2023-11-28T11:18:51.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 0x7fcfb8156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 0x7fcfb815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81f2a50 con 0x7fcfb80d2320 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81f2d30 con 0x7fcfb815c9b0 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.443+0000 7fcfbddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81f3010 con 0x7fcfb8161340 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60566/0 (socket says 172.21.15.53:60566) 2023-11-28T11:18:51.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 -- 172.21.15.53:0/2419712190 learned_addr learned my addr 172.21.15.53:0/2419712190 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb7fff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 0x7fcfb815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb6ffd700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 0x7fcfb8156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 -- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 msgr2=0x7fcfb815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 0x7fcfb815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 -- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 msgr2=0x7fcfb8156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 0x7fcfb8156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 -- 172.21.15.53:0/2419712190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0094a0 con 0x7fcfb80d2320 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb7fff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 0x7fcfb815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb6ffd700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 0x7fcfb8156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.444+0000 7fcfb77fe700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c4e0 tx=0x7fcfa800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.445+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80052d0 con 0x7fcfb80d2320 2023-11-28T11:18:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.445+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8005430 con 0x7fcfb80d2320 2023-11-28T11:18:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.445+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80038c0 con 0x7fcfb80d2320 2023-11-28T11:18:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.445+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb815cd80 con 0x7fcfb80d2320 2023-11-28T11:18:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.445+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb80d15e0 con 0x7fcfb80d2320 2023-11-28T11:18:51.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.446+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb815cf10 con 0x7fcfb80d2320 2023-11-28T11:18:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.447+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfa8019030 con 0x7fcfb80d2320 2023-11-28T11:18:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.447+0000 7fcf9ffff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 0x7fcfa406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.447+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcfa8084da0 con 0x7fcfb80d2320 2023-11-28T11:18:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.448+0000 7fcfb6ffd700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 0x7fcfa406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.449+0000 7fcfb6ffd700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 0x7fcfa406a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80b76c0 tx=0x7fcfac01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.451+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfa8051860 con 0x7fcfb80d2320 2023-11-28T11:18:51.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.734+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcfb80745c0 con 0x7fcfb80d2320 2023-11-28T11:18:51.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.735+0000 7fcf9ffff700 1 -- 172.21.15.53:0/2419712190 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fcfa8054e80 con 0x7fcfb80d2320 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVlNkJINxAAwB6S1pnMrZk3sG6i+8PR9Q==", 2023-11-28T11:18:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlHGGyFBAAkLNG+TdDzS8Rd+Q2LV6BYw==", 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlH5ZYCBAAdBiB6rthc32/VYiX/UUEpg==", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:18:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlUdNXIRAAete67odjh+SjYr5KUAmDvw==", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAjzGVlnYF9HxAAx5MdeV6qs7Ce0YL/BzjUuQ==", 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAmzGVlheXPJhAAqVOmYx93P2jBYNhzlsg4EQ==", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQApzGVlRXhLLhAAcCx5wY3VywAx4UWnz15sOg==", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAszGVlxc3CNBAAo8dwkn6w3PZYcNFkTdxZEA==", 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAxzGVlKUZYJRAALIbrf8b2mXrVTt8ebMFSnA==", 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA0zGVl2RxXKRAAyHpuvnKEAS2o9BNE1LdRGw==", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA3zGVlFw1WLRAAf+SrDy5UyYmdT3NfXFYA2A==", 2023-11-28T11:18:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA6zGVlCqCMMBAAq6pur3Ycnp4CiTD40y4upw==", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:18:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlvIIFLBAAxrqCs6enHKg93ih+QHQcdw==", 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVlkkNgABAAD7UONIytbl4yWdjvGODhXw==", 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlrXsWNBAA+cEN015xjMYKa6JKavZobQ==", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-11-28T11:18:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVl2rlMCBAA0jfD14UyD2AjgyjtkShrTw==", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAezGVlf022ORAA2Fv9xeE7q2Pkr4B6c7ncyA==", 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVlQL37LxAAuUJNxVFCtiCveOqUUd3axg==", 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:18:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVl/cz7LxAAm3nlkr0kDhR/c19d5JZo6w==", 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVletb7LxAAKZxt5cz/kk0aOXOKtdnXcA==", 2023-11-28T11:18:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVlnt/7LxAA85qJHjkfvl3OPuA011Fwfw==", 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVlE+n7LxAAhJO+k4cTefxl+vvDhAKX8Q==", 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQBHzGVl9fL7LxAA/8jid2yMrfjmY5UYunrHCw==", 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVllCcfLxAAjmT46vM+3CEOyE6aCbidrQ==", 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVl91F6JhAAGc79E5/TfWduB198lkdQqw==", 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-28T11:18:51.755 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:18:51.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 msgr2=0x7fcfa406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 0x7fcfa406a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80b76c0 tx=0x7fcfac01c040 comp rx=0 tx=0).stop 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 msgr2=0x7fcfb8157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c4e0 tx=0x7fcfa800c8a0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 shutdown_connections 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fcfa40682f0 0x7fcfa406a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb8161340 0x7fcfb815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb80d2320 0x7fcfb8157270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 --2- 172.21.15.53:0/2419712190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb815c9b0 0x7fcfb8156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 >> 172.21.15.53:0/2419712190 conn(0x7fcfb8072600 msgr2=0x7fcfb815a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 shutdown_connections 2023-11-28T11:18:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.737+0000 7fcfbddff700 1 -- 172.21.15.53:0/2419712190 wait complete. 2023-11-28T11:18:51.759 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:18:51.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.963+0000 7fddc9dff700 1 Processor -- start 2023-11-28T11:18:51.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.963+0000 7fddc9dff700 1 -- start start 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc9dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fddc4161340 0x7fddc40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 0x7fddc40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc9dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40d1950 con 0x7fddc415c9b0 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc9dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40b5800 con 0x7fddc4161340 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc9dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40b5300 con 0x7fddc40d2320 2023-11-28T11:18:51.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fddc4161340 0x7fddc40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fddc4161340 0x7fddc40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60580/0 (socket says 172.21.15.53:60580) 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 -- 172.21.15.53:0/4112142883 learned_addr learned my addr 172.21.15.53:0/4112142883 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc3fff700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 0x7fddc40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 -- 172.21.15.53:0/4112142883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 msgr2=0x7fddc40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 0x7fddc40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 -- 172.21.15.53:0/4112142883 >> v1:172.21.15.157:6789/0 conn(0x7fddc415c9b0 legacy=0x7fddc40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc2ffd700 1 -- 172.21.15.53:0/4112142883 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40b5520 con 0x7fddc4161340 2023-11-28T11:18:51.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.965+0000 7fddc3fff700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 0x7fddc40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:51.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc2ffd700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc40d1de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fddb80098f0 tx=0x7fddb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab56c8326f8a730a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc27fc700 1 -- 172.21.15.53:0/4112142883 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800b8a0 con 0x7fddc4161340 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc27fc700 1 -- 172.21.15.53:0/4112142883 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800ba00 con 0x7fddc4161340 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc27fc700 1 -- 172.21.15.53:0/4112142883 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800bd90 con 0x7fddc4161340 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc9dff700 1 -- 172.21.15.53:0/4112142883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 msgr2=0x7fddc40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.966+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc40d1de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fddb80098f0 tx=0x7fddb8009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 -- 172.21.15.53:0/4112142883 shutdown_connections 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc40d2320 0x7fddc40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4112142883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc40d1de0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 -- 172.21.15.53:0/4112142883 >> 172.21.15.53:0/4112142883 conn(0x7fddc4072600 msgr2=0x7fddc40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 -- 172.21.15.53:0/4112142883 shutdown_connections 2023-11-28T11:18:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.967+0000 7fddc9dff700 1 -- 172.21.15.53:0/4112142883 wait complete. 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.968+0000 7fddc9dff700 1 Processor -- start 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.968+0000 7fddc9dff700 1 -- start start 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc40d2320 0x7fddc4157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc415c9b0 0x7fddc415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41e8420 con 0x7fddc4161340 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40ce1e0 con 0x7fddc40d2320 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.969+0000 7fddc9dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41e74c0 con 0x7fddc415c9b0 2023-11-28T11:18:51.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60592/0 (socket says 172.21.15.53:60592) 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 -- 172.21.15.53:0/4196105651 learned_addr learned my addr 172.21.15.53:0/4196105651 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc2ffd700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc415c9b0 0x7fddc415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc37fe700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc40d2320 0x7fddc4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 -- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc415c9b0 msgr2=0x7fddc415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc415c9b0 0x7fddc415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 -- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc40d2320 msgr2=0x7fddc4157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc40d2320 0x7fddc4157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:51.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 -- 172.21.15.53:0/4196105651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009550 con 0x7fddc4161340 2023-11-28T11:18:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.970+0000 7fddc3fff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fddbc007e40 tx=0x7fddbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.971+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00dd20 con 0x7fddc4161340 2023-11-28T11:18:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.971+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc00f040 con 0x7fddc4161340 2023-11-28T11:18:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.971+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00dea0 con 0x7fddc4161340 2023-11-28T11:18:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.971+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc40d15e0 con 0x7fddc4161340 2023-11-28T11:18:51.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.971+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc40d1300 con 0x7fddc4161340 2023-11-28T11:18:51.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.972+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc415cd80 con 0x7fddc4161340 2023-11-28T11:18:51.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.973+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fddbc01c030 con 0x7fddc4161340 2023-11-28T11:18:51.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.974+0000 7fddabfff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 0x7fddb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:51.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.974+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fddbc085860 con 0x7fddc4161340 2023-11-28T11:18:51.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.974+0000 7fddc37fe700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 0x7fddb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:51.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.975+0000 7fddc37fe700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 0x7fddb006a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddc41f5f80 tx=0x7fddb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:51.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:51.978+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fddbc0522c0 con 0x7fddc4161340 2023-11-28T11:18:52.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.255+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fddc40745c0 con 0x7fddc4161340 2023-11-28T11:18:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.259+0000 7fddabfff700 1 -- 172.21.15.53:0/4196105651 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fddbc0558e0 con 0x7fddc4161340 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 msgr2=0x7fddb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 0x7fddb006a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddc41f5f80 tx=0x7fddb4009040 comp rx=0 tx=0).stop 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 msgr2=0x7fddc415ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fddbc007e40 tx=0x7fddbc00d3b0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 shutdown_connections 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fddb00682f0 0x7fddb006a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddc4161340 0x7fddc415ac50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc40d2320 0x7fddc4157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 --2- 172.21.15.53:0/4196105651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc415c9b0 0x7fddc415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 >> 172.21.15.53:0/4196105651 conn(0x7fddc4072600 msgr2=0x7fddc40b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:52.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 shutdown_connections 2023-11-28T11:18:52.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.262+0000 7fddc9dff700 1 -- 172.21.15.53:0/4196105651 wait complete. 2023-11-28T11:18:52.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:18:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.482+0000 7fa88bfff700 1 Processor -- start 2023-11-28T11:18:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.482+0000 7fa88bfff700 1 -- start start 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.483+0000 7fa88bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa88415c9b0 0x7fa8840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.483+0000 7fa88bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa884161340 0x7fa8840d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.483+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8840d2320 0x7fa8840cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.483+0000 7fa88bfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8840b5800 con 0x7fa884161340 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88bfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8840d1950 con 0x7fa88415c9b0 2023-11-28T11:18:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88bfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840b5520 con 0x7fa8840d2320 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa884161340 0x7fa8840d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa884161340 0x7fa8840d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60606/0 (socket says 172.21.15.53:60606) 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 -- 172.21.15.53:0/2571528025 learned_addr learned my addr 172.21.15.53:0/2571528025 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88a59c700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8840d2320 0x7fa8840cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa889d9b700 1 --2- 172.21.15.53:0/2571528025 >> v2:172.21.15.157:3300/0 conn(0x7fa88415c9b0 0x7fa8840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 -- 172.21.15.53:0/2571528025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8840d2320 msgr2=0x7fa8840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8840d2320 0x7fa8840cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 -- 172.21.15.53:0/2571528025 >> v2:172.21.15.157:3300/0 conn(0x7fa88415c9b0 msgr2=0x7fa8840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 --2- 172.21.15.53:0/2571528025 >> v2:172.21.15.157:3300/0 conn(0x7fa88415c9b0 0x7fa8840b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 -- 172.21.15.53:0/2571528025 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840d1300 con 0x7fa884161340 2023-11-28T11:18:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.484+0000 7fa88959a700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa884161340 0x7fa8840d1de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa884067460 tx=0x7fa874009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cdfcd1d7edf36d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa888d99700 1 -- 172.21.15.53:0/2571528025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400b870 con 0x7fa884161340 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa888d99700 1 -- 172.21.15.53:0/2571528025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400b9d0 con 0x7fa884161340 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa888d99700 1 -- 172.21.15.53:0/2571528025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400bd60 con 0x7fa884161340 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 -- 172.21.15.53:0/2571528025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa884161340 msgr2=0x7fa8840d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa884161340 0x7fa8840d1de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa884067460 tx=0x7fa874009bd0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 -- 172.21.15.53:0/2571528025 shutdown_connections 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8840d2320 0x7fa8840cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 --2- 172.21.15.53:0/2571528025 >> v2:172.21.15.157:3300/0 conn(0x7fa88415c9b0 0x7fa8840b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 --2- 172.21.15.53:0/2571528025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa884161340 0x7fa8840d1de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 -- 172.21.15.53:0/2571528025 >> 172.21.15.53:0/2571528025 conn(0x7fa884072600 msgr2=0x7fa8840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.485+0000 7fa88bfff700 1 -- 172.21.15.53:0/2571528025 shutdown_connections 2023-11-28T11:18:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.486+0000 7fa88bfff700 1 -- 172.21.15.53:0/2571528025 wait complete. 2023-11-28T11:18:52.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.487+0000 7fa88bfff700 1 Processor -- start 2023-11-28T11:18:52.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.487+0000 7fa88bfff700 1 -- start start 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 0x7fa88415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 0x7fa88415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884159410 con 0x7fa8840d2320 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884155a40 con 0x7fa884161340 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.488+0000 7fa88bfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa884155d20 con 0x7fa88415c9b0 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60610/0 (socket says 172.21.15.53:60610) 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 -- 172.21.15.53:0/660628766 learned_addr learned my addr 172.21.15.53:0/660628766 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa88959a700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 0x7fa88415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 -- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 msgr2=0x7fa88415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 0x7fa88415f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa88a59c700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 0x7fa88415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 -- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 msgr2=0x7fa88415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 0x7fa88415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 -- 172.21.15.53:0/660628766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874009550 con 0x7fa8840d2320 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa88959a700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 0x7fa88415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa88a59c700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 0x7fa88415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.489+0000 7fa889d9b700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e810 tx=0x7fa88000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.490+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880013610 con 0x7fa8840d2320 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.490+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880013770 con 0x7fa8840d2320 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.490+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880013b20 con 0x7fa8840d2320 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.490+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8840b5130 con 0x7fa8840d2320 2023-11-28T11:18:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.490+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8840d15e0 con 0x7fa8840d2320 2023-11-28T11:18:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.491+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8840b52c0 con 0x7fa8840d2320 2023-11-28T11:18:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.492+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa880023070 con 0x7fa8840d2320 2023-11-28T11:18:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.493+0000 7fa87a7fc700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.493+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa8800856b0 con 0x7fa8840d2320 2023-11-28T11:18:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.493+0000 7fa88959a700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.494+0000 7fa88959a700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 0x7fa86806a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8841e8e20 tx=0x7fa874008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.496+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8800520f0 con 0x7fa8840d2320 2023-11-28T11:18:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.782+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fa8840745c0 con 0x7fa8840d2320 2023-11-28T11:18:52.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.786+0000 7fa87a7fc700 1 -- 172.21.15.53:0/660628766 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa880055710 con 0x7fa8840d2320 2023-11-28T11:18:52.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 msgr2=0x7fa86806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 0x7fa86806a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8841e8e20 tx=0x7fa874008030 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 msgr2=0x7fa884155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e810 tx=0x7fa88000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 shutdown_connections 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884161340 0x7fa88415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8840d2320 0x7fa884155080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 --2- 172.21.15.53:0/660628766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88415c9b0 0x7fa88415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 >> 172.21.15.53:0/660628766 conn(0x7fa884072600 msgr2=0x7fa88415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 shutdown_connections 2023-11-28T11:18:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:52.788+0000 7fa88bfff700 1 -- 172.21.15.53:0/660628766 wait complete. 2023-11-28T11:18:52.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:18:53.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.011+0000 7faa495ff700 1 Processor -- start 2023-11-28T11:18:53.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.011+0000 7faa495ff700 1 -- start start 2023-11-28T11:18:53.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.012+0000 7faa495ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa440cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.012+0000 7faa495ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440d1950 con 0x7faa4415c9b0 2023-11-28T11:18:53.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.012+0000 7faa495ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440ce140 con 0x7faa44161340 2023-11-28T11:18:53.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.012+0000 7faa495ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b5800 con 0x7faa440cd7c0 2023-11-28T11:18:53.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa42ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faa4415c9b0 0x7faa440b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40000/0 (socket says 172.21.15.53:40000) 2023-11-28T11:18:53.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa42ffd700 1 -- 172.21.15.53:0/3015544165 learned_addr learned my addr 172.21.15.53:0/3015544165 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:53.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa437fe700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa440cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426304949 0 0) 0x7faa440d1950 con 0x7faa4415c9b0 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa300035b0 con 0x7faa4415c9b0 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386968330 0 0) 0x7faa440ce140 con 0x7faa44161340 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa440d1950 con 0x7faa44161340 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214297425 0 0) 0x7faa300035b0 con 0x7faa4415c9b0 2023-11-28T11:18:53.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 msgr2=0x7faa440cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:53.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa440cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.013+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 >> v1:172.21.15.157:6789/0 conn(0x7faa44161340 legacy=0x7faa440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440d24f0 con 0x7faa4415c9b0 2023-11-28T11:18:53.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7faa34001670 con 0x7faa4415c9b0 2023-11-28T11:18:53.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 >> v1:172.21.15.53:6789/0 conn(0x7faa4415c9b0 legacy=0x7faa440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa30003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa30004120 0x7faa30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004d90 0x7faa30005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b5800 con 0x7faa30004120 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000bba0 con 0x7faa30004d90 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000bd30 con 0x7faa440cd7c0 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa30004120 0x7faa30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa42ffd700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa30003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa437fe700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004d90 0x7faa30005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 -- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 msgr2=0x7faa30003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:53.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440cd7c0 0x7faa30003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 -- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004d90 msgr2=0x7faa30005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004d90 0x7faa30005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.014+0000 7faa3bfff700 1 -- 172.21.15.53:0/3015544165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300035b0 con 0x7faa30004120 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa3bfff700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa30004120 0x7faa30004850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001940 tx=0x7faa30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23eaac23781f9d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c001430 con 0x7faa30004120 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c001d70 con 0x7faa30004120 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa427fc700 1 -- 172.21.15.53:0/3015544165 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c005510 con 0x7faa30004120 2023-11-28T11:18:53.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa495ff700 1 -- 172.21.15.53:0/3015544165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa30004120 msgr2=0x7faa30004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa495ff700 1 --2- 172.21.15.53:0/3015544165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa30004120 0x7faa30004850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001940 tx=0x7faa30005880 comp rx=0 tx=0).stop 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.015+0000 7faa437fe700 1 -- 172.21.15.53:0/3015544165 reap_dead start 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.016+0000 7faa495ff700 1 -- 172.21.15.53:0/3015544165 shutdown_connections 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.016+0000 7faa495ff700 1 -- 172.21.15.53:0/3015544165 >> 172.21.15.53:0/3015544165 conn(0x7faa44072600 msgr2=0x7faa440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.016+0000 7faa495ff700 1 -- 172.21.15.53:0/3015544165 shutdown_connections 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.016+0000 7faa495ff700 1 -- 172.21.15.53:0/3015544165 wait complete. 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.017+0000 7faa495ff700 1 Processor -- start 2023-11-28T11:18:53.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.018+0000 7faa495ff700 1 -- start start 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.018+0000 7faa495ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 0x7faa44156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa495ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 0x7faa440cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa495ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa495ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440cd680 con 0x7faa440c6ca0 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa495ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440cd3a0 con 0x7faa30004120 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa495ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa441e7950 con 0x7faa440ccee0 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60640/0 (socket says 172.21.15.53:60640) 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 -- 172.21.15.53:0/1835122515 learned_addr learned my addr 172.21.15.53:0/1835122515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa42ffd700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 0x7faa44156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 -- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 msgr2=0x7faa440cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa3bfff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 0x7faa440cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 0x7faa440cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 -- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 msgr2=0x7faa44156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 0x7faa44156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 -- 172.21.15.53:0/1835122515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440d1300 con 0x7faa440c6ca0 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa42ffd700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 0x7faa44156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa3bfff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 0x7faa440cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.019+0000 7faa437fe700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faa4406bbb0 tx=0x7faa3c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.020+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00e070 con 0x7faa440c6ca0 2023-11-28T11:18:53.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.020+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c003c30 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.020+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440b5520 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.020+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003de0 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.020+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa440d15e0 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.021+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa441e77b0 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.022+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa3c011a10 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.023+0000 7faa3b7fe700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 0x7faa2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.023+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa3c085ee0 con 0x7faa440c6ca0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.023+0000 7faa42ffd700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 0x7faa2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.024+0000 7faa42ffd700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 0x7faa2406a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa441e7670 tx=0x7faa34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.027+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa441e77b0 con 0x7faa440c6ca0 2023-11-28T11:18:53.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.311+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7faa441e77b0 con 0x7faa440c6ca0 2023-11-28T11:18:53.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.315+0000 7faa3b7fe700 1 -- 172.21.15.53:0/1835122515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7faa441e77b0 con 0x7faa440c6ca0 2023-11-28T11:18:53.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 msgr2=0x7faa2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 0x7faa2406a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa441e7670 tx=0x7faa34009040 comp rx=0 tx=0).stop 2023-11-28T11:18:53.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 msgr2=0x7faa4415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faa4406bbb0 tx=0x7faa3c0114b0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 shutdown_connections 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa240682f0 0x7faa2406a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa440c6ca0 0x7faa4415e350 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa30004120 0x7faa44156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 --2- 172.21.15.53:0/1835122515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa440ccee0 0x7faa440cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 >> 172.21.15.53:0/1835122515 conn(0x7faa44072600 msgr2=0x7faa4415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 shutdown_connections 2023-11-28T11:18:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.317+0000 7faa495ff700 1 -- 172.21.15.53:0/1835122515 wait complete. 2023-11-28T11:18:53.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:18:53.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.537+0000 7f69473ff700 1 Processor -- start 2023-11-28T11:18:53.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.538+0000 7f69473ff700 1 -- start start 2023-11-28T11:18:53.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f69473ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f694015c9b0 0x7f69400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f69473ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69400cd7c0 0x7f69400cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f69473ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69400d1950 con 0x7f6940161340 2023-11-28T11:18:53.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f69473ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400b5800 con 0x7f69400cd7c0 2023-11-28T11:18:53.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f69473ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400ce140 con 0x7f694015c9b0 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694599c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69400cd7c0 0x7f69400cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694499a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6940161340 0x7f69400d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40006/0 (socket says 172.21.15.53:40006) 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694499a700 1 -- 172.21.15.53:0/930982547 learned_addr learned my addr 172.21.15.53:0/930982547 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694519b700 1 --2- 172.21.15.53:0/930982547 >> v2:172.21.15.157:3300/0 conn(0x7f694015c9b0 0x7f69400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694599c700 1 -- 172.21.15.53:0/930982547 >> v2:172.21.15.157:3300/0 conn(0x7f694015c9b0 msgr2=0x7f69400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694599c700 1 --2- 172.21.15.53:0/930982547 >> v2:172.21.15.157:3300/0 conn(0x7f694015c9b0 0x7f69400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694599c700 1 -- 172.21.15.53:0/930982547 >> v1:172.21.15.53:6789/0 conn(0x7f6940161340 legacy=0x7f69400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f694599c700 1 -- 172.21.15.53:0/930982547 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400d24f0 con 0x7f69400cd7c0 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.539+0000 7f6937fff700 1 -- 172.21.15.53:0/930982547 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673875219 0 0) 0x7f69400d1950 con 0x7f6940161340 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.540+0000 7f694599c700 1 --2- 172.21.15.53:0/930982547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69400cdc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6940067460 tx=0x7f693800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abb192777f6e62d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.540+0000 7f6937fff700 1 -- 172.21.15.53:0/930982547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800e070 con 0x7f69400cd7c0 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.540+0000 7f6937fff700 1 -- 172.21.15.53:0/930982547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693800bbd0 con 0x7f69400cd7c0 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.540+0000 7f6937fff700 1 -- 172.21.15.53:0/930982547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938012430 con 0x7f69400cd7c0 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.540+0000 7f69473ff700 1 -- 172.21.15.53:0/930982547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 msgr2=0x7f69400cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 --2- 172.21.15.53:0/930982547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69400cdc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6940067460 tx=0x7f693800b330 comp rx=0 tx=0).stop 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 -- 172.21.15.53:0/930982547 shutdown_connections 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 --2- 172.21.15.53:0/930982547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69400cdc00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 --2- 172.21.15.53:0/930982547 >> v2:172.21.15.157:3300/0 conn(0x7f694015c9b0 0x7f69400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 -- 172.21.15.53:0/930982547 >> 172.21.15.53:0/930982547 conn(0x7f6940072600 msgr2=0x7f69400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 -- 172.21.15.53:0/930982547 shutdown_connections 2023-11-28T11:18:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.541+0000 7f69473ff700 1 -- 172.21.15.53:0/930982547 wait complete. 2023-11-28T11:18:53.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.542+0000 7f69473ff700 1 Processor -- start 2023-11-28T11:18:53.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.543+0000 7f69473ff700 1 -- start start 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 0x7f69400ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6940161340 0x7f6940156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940157b00 con 0x7f69400cd7c0 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400ca780 con 0x7f6940161340 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f69473ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6940157060 con 0x7f694015c9b0 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60666/0 (socket says 172.21.15.53:60666) 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 -- 172.21.15.53:0/3548988233 learned_addr learned my addr 172.21.15.53:0/3548988233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:53.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694599c700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6940161340 0x7f6940156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694499a700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 0x7f69400ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 -- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 msgr2=0x7f69400ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 0x7f69400ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 -- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6940161340 msgr2=0x7f6940156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6940161340 0x7f6940156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694519b700 1 -- 172.21.15.53:0/3548988233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938009a70 con 0x7f69400cd7c0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.544+0000 7f694499a700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 0x7f69400ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f694519b700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d800 tx=0x7f693c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0125f0 con 0x7f69400cd7c0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c012750 con 0x7f69400cd7c0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69400d1300 con 0x7f69400cd7c0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c012b00 con 0x7f69400cd7c0 2023-11-28T11:18:53.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.545+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69400b5520 con 0x7f69400cd7c0 2023-11-28T11:18:53.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.546+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69400d15e0 con 0x7f69400cd7c0 2023-11-28T11:18:53.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.547+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f693c012c60 con 0x7f69400cd7c0 2023-11-28T11:18:53.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.548+0000 7f6935ffb700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 0x7f692406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:53.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.548+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f693c085e30 con 0x7f69400cd7c0 2023-11-28T11:18:53.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.548+0000 7f694499a700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 0x7f692406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:53.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.549+0000 7f694499a700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 0x7f692406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69401f5bf0 tx=0x7f6930009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.552+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f693c0529e0 con 0x7f69400cd7c0 2023-11-28T11:18:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.829+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f69400745c0 con 0x7f69400cd7c0 2023-11-28T11:18:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.833+0000 7f6935ffb700 1 -- 172.21.15.53:0/3548988233 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f693c056000 con 0x7f69400cd7c0 2023-11-28T11:18:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 msgr2=0x7f692406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 0x7f692406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69401f5bf0 tx=0x7f6930009040 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 msgr2=0x7f69401571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d800 tx=0x7f693c00dbc0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 shutdown_connections 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69240682f0 0x7f692406a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6940161340 0x7f6940156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69400cd7c0 0x7f69401571f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 --2- 172.21.15.53:0/3548988233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f694015c9b0 0x7f69400ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 >> 172.21.15.53:0/3548988233 conn(0x7f6940072600 msgr2=0x7f694015a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 shutdown_connections 2023-11-28T11:18:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:53.835+0000 7f69473ff700 1 -- 172.21.15.53:0/3548988233 wait complete. 2023-11-28T11:18:53.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:18:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.056+0000 7fafad3ff700 1 Processor -- start 2023-11-28T11:18:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.057+0000 7fafad3ff700 1 -- start start 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7fafad3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fafa815c9b0 0x7fafa80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7fafad3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa80d1950 con 0x7fafa8161340 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7fafad3ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa80d24f0 con 0x7fafa80cd7c0 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7fafad3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa80b5800 con 0x7fafa815c9b0 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7faf9ffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fafa8161340 0x7fafa80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40018/0 (socket says 172.21.15.53:40018) 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7fafa6ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fafa815c9b0 0x7fafa80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.058+0000 7faf9ffff700 1 -- 172.21.15.53:0/1156494281 learned_addr learned my addr 172.21.15.53:0/1156494281 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa67fc700 1 -- 172.21.15.53:0/1156494281 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258933206 0 0) 0x7fafa80d1950 con 0x7fafa8161340 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa67fc700 1 -- 172.21.15.53:0/1156494281 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf940035b0 con 0x7fafa8161340 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa6ffd700 1 -- 172.21.15.53:0/1156494281 >> v1:172.21.15.157:6789/0 conn(0x7fafa80cd7c0 legacy=0x7fafa80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa6ffd700 1 -- 172.21.15.53:0/1156494281 >> v1:172.21.15.53:6789/0 conn(0x7fafa8161340 legacy=0x7fafa80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa6ffd700 1 -- 172.21.15.53:0/1156494281 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80b5520 con 0x7fafa815c9b0 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa6ffd700 1 --2- 172.21.15.53:0/1156494281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa815c9b0 0x7fafa80b4bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf98009840 tx=0x7faf98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a622bf02cdcc35d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa67fc700 1 -- 172.21.15.53:0/1156494281 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011830 con 0x7fafa815c9b0 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa67fc700 1 -- 172.21.15.53:0/1156494281 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98011990 con 0x7fafa815c9b0 2023-11-28T11:18:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.059+0000 7fafa67fc700 1 -- 172.21.15.53:0/1156494281 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011d20 con 0x7fafa815c9b0 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 -- 172.21.15.53:0/1156494281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa815c9b0 msgr2=0x7fafa80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 --2- 172.21.15.53:0/1156494281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa815c9b0 0x7fafa80b4bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf98009840 tx=0x7faf98009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 -- 172.21.15.53:0/1156494281 shutdown_connections 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 --2- 172.21.15.53:0/1156494281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa815c9b0 0x7fafa80b4bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 -- 172.21.15.53:0/1156494281 >> 172.21.15.53:0/1156494281 conn(0x7fafa8072600 msgr2=0x7fafa80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 -- 172.21.15.53:0/1156494281 shutdown_connections 2023-11-28T11:18:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.060+0000 7fafad3ff700 1 -- 172.21.15.53:0/1156494281 wait complete. 2023-11-28T11:18:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.062+0000 7fafad3ff700 1 Processor -- start 2023-11-28T11:18:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.062+0000 7fafad3ff700 1 -- start start 2023-11-28T11:18:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa80cd7c0 0x7fafa814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafa815c9b0 0x7fafa81e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa814abc0 con 0x7fafa8161340 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa815deb0 con 0x7fafa80cd7c0 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafad3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa815e190 con 0x7fafa815c9b0 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafa77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafa77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60670/0 (socket says 172.21.15.53:60670) 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafa77fe700 1 -- 172.21.15.53:0/2648370871 learned_addr learned my addr 172.21.15.53:0/2648370871 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7fafa6ffd700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa80cd7c0 0x7fafa814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.063+0000 7faf9ffff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafa815c9b0 0x7fafa81e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 -- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafa815c9b0 msgr2=0x7fafa81e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafa815c9b0 0x7fafa81e77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 -- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa80cd7c0 msgr2=0x7fafa814a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa80cd7c0 0x7fafa814a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 -- 172.21.15.53:0/2648370871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980094a0 con 0x7fafa8161340 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafa77fe700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fafa000cb50 tx=0x7fafa000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0003c00 con 0x7fafa8161340 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0003d60 con 0x7fafa8161340 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa000aa20 con 0x7fafa8161340 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa80d15e0 con 0x7fafa8161340 2023-11-28T11:18:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.064+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa80d1300 con 0x7fafa8161340 2023-11-28T11:18:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.065+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa80b5130 con 0x7fafa8161340 2023-11-28T11:18:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.066+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fafa0010050 con 0x7fafa8161340 2023-11-28T11:18:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.067+0000 7faf9f7fe700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 0x7faf8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.067+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fafa00857d0 con 0x7fafa8161340 2023-11-28T11:18:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.068+0000 7fafa6ffd700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 0x7faf8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.068+0000 7fafa6ffd700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 0x7faf8806a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf980090b0 tx=0x7faf9801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.071+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fafa0052380 con 0x7fafa8161340 2023-11-28T11:18:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.354+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fafa80745c0 con 0x7fafa8161340 2023-11-28T11:18:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.358+0000 7faf9f7fe700 1 -- 172.21.15.53:0/2648370871 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fafa00559a0 con 0x7fafa8161340 2023-11-28T11:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 msgr2=0x7faf8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 0x7faf8806a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf980090b0 tx=0x7faf9801c040 comp rx=0 tx=0).stop 2023-11-28T11:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 msgr2=0x7fafa8160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fafa000cb50 tx=0x7fafa000ce60 comp rx=0 tx=0).stop 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 shutdown_connections 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf880682f0 0x7faf8806a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa8161340 0x7fafa8160460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa80cd7c0 0x7fafa814a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 --2- 172.21.15.53:0/2648370871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafa815c9b0 0x7fafa81e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 >> 172.21.15.53:0/2648370871 conn(0x7fafa8072600 msgr2=0x7fafa80b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 shutdown_connections 2023-11-28T11:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.360+0000 7fafad3ff700 1 -- 172.21.15.53:0/2648370871 wait complete. 2023-11-28T11:18:54.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:18:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.581+0000 7ff918bff700 1 Processor -- start 2023-11-28T11:18:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.581+0000 7ff918bff700 1 -- start start 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff918bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff914161340 0x7ff9140d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff918bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9140cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff918bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9140d1950 con 0x7ff91415c9b0 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff918bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140b5800 con 0x7ff914161340 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff918bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140b5300 con 0x7ff9140d2320 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff90bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff914161340 0x7ff9140d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.582+0000 7ff90bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff914161340 0x7ff9140d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60686/0 (socket says 172.21.15.53:60686) 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 -- 172.21.15.53:0/292792198 learned_addr learned my addr 172.21.15.53:0/292792198 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:54.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff912d9d700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9140cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 -- 172.21.15.53:0/292792198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 msgr2=0x7ff9140cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9140cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 -- 172.21.15.53:0/292792198 >> v1:172.21.15.157:6789/0 conn(0x7ff91415c9b0 legacy=0x7ff9140b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 -- 172.21.15.53:0/292792198 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140b5520 con 0x7ff914161340 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff912d9d700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9140cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:54.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff90bfff700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff914161340 0x7ff9140d1de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0098f0 tx=0x7ff8fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b83e0967011f13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff911d9b700 1 -- 172.21.15.53:0/292792198 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00b8a0 con 0x7ff914161340 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.583+0000 7ff911d9b700 1 -- 172.21.15.53:0/292792198 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00ba00 con 0x7ff914161340 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff911d9b700 1 -- 172.21.15.53:0/292792198 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00bd90 con 0x7ff914161340 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 -- 172.21.15.53:0/292792198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff914161340 msgr2=0x7ff9140d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff914161340 0x7ff9140d1de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0098f0 tx=0x7ff8fc009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 -- 172.21.15.53:0/292792198 shutdown_connections 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9140cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 --2- 172.21.15.53:0/292792198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff914161340 0x7ff9140d1de0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 -- 172.21.15.53:0/292792198 >> 172.21.15.53:0/292792198 conn(0x7ff914072600 msgr2=0x7ff9140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:54.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 -- 172.21.15.53:0/292792198 shutdown_connections 2023-11-28T11:18:54.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.584+0000 7ff918bff700 1 -- 172.21.15.53:0/292792198 wait complete. 2023-11-28T11:18:54.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.586+0000 7ff918bff700 1 Processor -- start 2023-11-28T11:18:54.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.586+0000 7ff918bff700 1 -- start start 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9141e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914161340 0x7ff9141e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91414ab30 con 0x7ff91415c9b0 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141e82c0 con 0x7ff914161340 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff918bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141e6500 con 0x7ff9140d2320 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60696/0 (socket says 172.21.15.53:60696) 2023-11-28T11:18:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 -- 172.21.15.53:0/704278761 learned_addr learned my addr 172.21.15.53:0/704278761 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff912d9d700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914161340 0x7ff9141e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff91259c700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9141e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 -- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 msgr2=0x7ff9141e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9141e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 -- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914161340 msgr2=0x7ff9141e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914161340 0x7ff9141e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.587+0000 7ff90bfff700 1 -- 172.21.15.53:0/704278761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc009550 con 0x7ff91415c9b0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff90bfff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000c00 tx=0x7ff8fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004480 con 0x7ff91415c9b0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc01f070 con 0x7ff91415c9b0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9140d15e0 con 0x7ff91415c9b0 2023-11-28T11:18:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0038c0 con 0x7ff91415c9b0 2023-11-28T11:18:54.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.588+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9140d1300 con 0x7ff91415c9b0 2023-11-28T11:18:54.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.589+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91415cd80 con 0x7ff91415c9b0 2023-11-28T11:18:54.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.590+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff8fc02a020 con 0x7ff91415c9b0 2023-11-28T11:18:54.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.591+0000 7ff90affd700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 0x7ff8f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:54.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.591+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff8fc087170 con 0x7ff91415c9b0 2023-11-28T11:18:54.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.591+0000 7ff91259c700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 0x7ff8f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:54.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.592+0000 7ff91259c700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 0x7ff8f406a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff904000f60 tx=0x7ff904009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:54.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.595+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff8fc053c30 con 0x7ff91415c9b0 2023-11-28T11:18:54.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.876+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ff9140745c0 con 0x7ff91415c9b0 2023-11-28T11:18:54.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.880+0000 7ff90affd700 1 -- 172.21.15.53:0/704278761 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v13) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff8fc057250 con 0x7ff91415c9b0 2023-11-28T11:18:54.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.882+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 msgr2=0x7ff8f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.882+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 0x7ff8f406a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff904000f60 tx=0x7ff904009040 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.882+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 msgr2=0x7ff91414a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.882+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000c00 tx=0x7ff8fc004750 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 shutdown_connections 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8f40682f0 0x7ff8f406a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9140d2320 0x7ff9141e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff914161340 0x7ff9141e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 --2- 172.21.15.53:0/704278761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91415c9b0 0x7ff91414a5f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 >> 172.21.15.53:0/704278761 conn(0x7ff914072600 msgr2=0x7ff9140b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 shutdown_connections 2023-11-28T11:18:54.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:54.883+0000 7ff918bff700 1 -- 172.21.15.53:0/704278761 wait complete. 2023-11-28T11:18:54.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.102+0000 7f5c68fff700 1 Processor -- start 2023-11-28T11:18:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.102+0000 7f5c68fff700 1 -- start start 2023-11-28T11:18:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.103+0000 7f5c68fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6415c9b0 0x7f5c640b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.103+0000 7f5c68fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c64161340 0x7f5c640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.103+0000 7f5c68fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c640d1950 con 0x7f5c640d2320 2023-11-28T11:18:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.103+0000 7f5c68fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c640b5800 con 0x7f5c64161340 2023-11-28T11:18:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.103+0000 7f5c68fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b5520 con 0x7f5c6415c9b0 2023-11-28T11:18:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c64161340 0x7f5c640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c6259c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6415c9b0 0x7f5c640b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c64161340 0x7f5c640d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:54042/0 (socket says 172.21.15.53:54042) 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 -- 172.21.15.53:0/348758407 learned_addr learned my addr 172.21.15.53:0/348758407 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c62d9d700 1 --1- 172.21.15.53:0/348758407 >> v1:172.21.15.157:6789/0 conn(0x7f5c640d2320 0x7f5c640cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:51002/0 (socket says 172.21.15.53:51002) 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 -- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6415c9b0 msgr2=0x7f5c640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 --2- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6415c9b0 0x7f5c640b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.104+0000 7f5c5bfff700 1 -- 172.21.15.53:0/348758407 >> v1:172.21.15.157:6789/0 conn(0x7f5c640d2320 legacy=0x7f5c640cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.105+0000 7f5c5bfff700 1 -- 172.21.15.53:0/348758407 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640d1300 con 0x7f5c64161340 2023-11-28T11:18:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.105+0000 7f5c5bfff700 1 --2- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c640d1de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0098f0 tx=0x7f5c4c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e937bf40f489d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.105+0000 7f5c61d9b700 1 -- 172.21.15.53:0/348758407 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011830 con 0x7f5c64161340 2023-11-28T11:18:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.105+0000 7f5c61d9b700 1 -- 172.21.15.53:0/348758407 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c011990 con 0x7f5c64161340 2023-11-28T11:18:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.105+0000 7f5c61d9b700 1 -- 172.21.15.53:0/348758407 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011d20 con 0x7f5c64161340 2023-11-28T11:18:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 -- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 msgr2=0x7f5c640d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 --2- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c640d1de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0098f0 tx=0x7f5c4c009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 -- 172.21.15.53:0/348758407 shutdown_connections 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 --2- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6415c9b0 0x7f5c640b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 --2- 172.21.15.53:0/348758407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c640d1de0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 -- 172.21.15.53:0/348758407 >> 172.21.15.53:0/348758407 conn(0x7f5c64072600 msgr2=0x7f5c640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 -- 172.21.15.53:0/348758407 shutdown_connections 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.106+0000 7f5c68fff700 1 -- 172.21.15.53:0/348758407 wait complete. 2023-11-28T11:18:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.108+0000 7f5c68fff700 1 Processor -- start 2023-11-28T11:18:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.108+0000 7f5c68fff700 1 -- start start 2023-11-28T11:18:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c640d2320 0x7f5c64153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c6415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64156640 con 0x7f5c6415c9b0 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64153c20 con 0x7f5c64161340 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c68fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640c6480 con 0x7f5c640d2320 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60702/0 (socket says 172.21.15.53:60702) 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c5bfff700 1 -- 172.21.15.53:0/1431496159 learned_addr learned my addr 172.21.15.53:0/1431496159 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c6259c700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c640d2320 0x7f5c64153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.109+0000 7f5c62d9d700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c6415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 -- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c640d2320 msgr2=0x7f5c64153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c640d2320 0x7f5c64153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 -- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 msgr2=0x7f5c6415ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c6415ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 -- 172.21.15.53:0/1431496159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c009550 con 0x7f5c6415c9b0 2023-11-28T11:18:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c62d9d700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c6415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5bfff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009c40 tx=0x7f5c4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c004480 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c01f070 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c640b5130 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0038c0 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.110+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c640d15e0 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.111+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c640b52c0 con 0x7f5c6415c9b0 2023-11-28T11:18:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.112+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c4c02a020 con 0x7f5c6415c9b0 2023-11-28T11:18:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.114+0000 7f5c5affd700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 0x7f5c4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.114+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5c4c086f90 con 0x7f5c6415c9b0 2023-11-28T11:18:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.114+0000 7f5c6259c700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 0x7f5c4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.115+0000 7f5c6259c700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 0x7f5c4406a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009be0 tx=0x7f5c54009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.118+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c4c053a50 con 0x7f5c6415c9b0 2023-11-28T11:18:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.324+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c640745c0 con 0x7f5c6415c9b0 2023-11-28T11:18:55.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.325+0000 7f5c5affd700 1 -- 172.21.15.53:0/1431496159 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+14267 (secure 0 0 0) 0x7f5c4c015090 con 0x7f5c6415c9b0 2023-11-28T11:18:55.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:55.326 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":32,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:51.206653+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.327+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 msgr2=0x7f5c4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.327+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 0x7f5c4406a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009be0 tx=0x7f5c54009210 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.327+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 msgr2=0x7f5c640c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.327+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009c40 tx=0x7f5c4c004750 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 shutdown_connections 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5c440682f0 0x7f5c4406a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c640d2320 0x7f5c64153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c64161340 0x7f5c6415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 --2- 172.21.15.53:0/1431496159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c6415c9b0 0x7f5c640c5f40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 >> 172.21.15.53:0/1431496159 conn(0x7f5c64072600 msgr2=0x7f5c6415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:55.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 shutdown_connections 2023-11-28T11:18:55.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.328+0000 7f5c68fff700 1 -- 172.21.15.53:0/1431496159 wait complete. 2023-11-28T11:18:55.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:18:55.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:18:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.552+0000 7fecaa3ff700 1 Processor -- start 2023-11-28T11:18:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.552+0000 7fecaa3ff700 1 -- start start 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7fecaa3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 0x7feca40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7fecaa3ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feca4161340 0x7feca40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7fecaa3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca40d1950 con 0x7feca40d2320 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7fecaa3ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca40b5800 con 0x7feca4161340 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7fecaa3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca40b5520 con 0x7feca415c9b0 2023-11-28T11:18:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7feca899c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feca40d2320 0x7feca40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40022/0 (socket says 172.21.15.53:40022) 2023-11-28T11:18:55.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.553+0000 7feca899c700 1 -- 172.21.15.53:0/1506769753 learned_addr learned my addr 172.21.15.53:0/1506769753 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:55.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 --2- 172.21.15.53:0/1506769753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 0x7feca40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca37fe700 1 --2- 172.21.15.53:0/1506769753 >> v2:172.21.15.157:3300/0 conn(0x7feca4161340 0x7feca40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca2ffd700 1 -- 172.21.15.53:0/1506769753 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438017107 0 0) 0x7feca40d1950 con 0x7feca40d2320 2023-11-28T11:18:55.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca2ffd700 1 -- 172.21.15.53:0/1506769753 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec8c0035b0 con 0x7feca40d2320 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 -- 172.21.15.53:0/1506769753 >> v2:172.21.15.157:3300/0 conn(0x7feca4161340 msgr2=0x7feca40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 --2- 172.21.15.53:0/1506769753 >> v2:172.21.15.157:3300/0 conn(0x7feca4161340 0x7feca40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 -- 172.21.15.53:0/1506769753 >> v1:172.21.15.53:6789/0 conn(0x7feca40d2320 legacy=0x7feca40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 -- 172.21.15.53:0/1506769753 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca40d1300 con 0x7feca415c9b0 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.554+0000 7feca3fff700 1 --2- 172.21.15.53:0/1506769753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 0x7feca40b4bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec94009840 tx=0x7fec94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9202a1480d4cd820 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7feca2ffd700 1 -- 172.21.15.53:0/1506769753 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94011830 con 0x7feca415c9b0 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7feca2ffd700 1 -- 172.21.15.53:0/1506769753 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94011990 con 0x7feca415c9b0 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7feca2ffd700 1 -- 172.21.15.53:0/1506769753 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94011d20 con 0x7feca415c9b0 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 -- 172.21.15.53:0/1506769753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 msgr2=0x7feca40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/1506769753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 0x7feca40b4bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec94009840 tx=0x7fec94009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 -- 172.21.15.53:0/1506769753 shutdown_connections 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/1506769753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca415c9b0 0x7feca40b4bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/1506769753 >> v2:172.21.15.157:3300/0 conn(0x7feca4161340 0x7feca40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 -- 172.21.15.53:0/1506769753 >> 172.21.15.53:0/1506769753 conn(0x7feca4072600 msgr2=0x7feca40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 -- 172.21.15.53:0/1506769753 shutdown_connections 2023-11-28T11:18:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.555+0000 7fecaa3ff700 1 -- 172.21.15.53:0/1506769753 wait complete. 2023-11-28T11:18:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.557+0000 7fecaa3ff700 1 Processor -- start 2023-11-28T11:18:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.557+0000 7fecaa3ff700 1 -- start start 2023-11-28T11:18:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 0x7feca41f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 0x7feca415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41514e0 con 0x7feca40d2320 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41f6dc0 con 0x7feca415c9b0 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.558+0000 7fecaa3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca41f70a0 con 0x7feca4161340 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60706/0 (socket says 172.21.15.53:60706) 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 -- 172.21.15.53:0/334285252 learned_addr learned my addr 172.21.15.53:0/334285252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca899c700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 0x7feca415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 -- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 msgr2=0x7feca415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca37fe700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 0x7feca41f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 0x7feca415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 -- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 msgr2=0x7feca41f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 0x7feca41f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 -- 172.21.15.53:0/334285252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940094a0 con 0x7feca40d2320 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca899c700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 0x7feca415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca37fe700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 0x7feca41f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca3fff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec940090b0 tx=0x7fec94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec940036a0 con 0x7feca40d2320 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9401f070 con 0x7feca40d2320 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.559+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94003a40 con 0x7feca40d2320 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.560+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca40b5130 con 0x7feca40d2320 2023-11-28T11:18:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.560+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca40d15e0 con 0x7feca40d2320 2023-11-28T11:18:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.561+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca40b52c0 con 0x7feca40d2320 2023-11-28T11:18:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.562+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec9400f050 con 0x7feca40d2320 2023-11-28T11:18:55.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.563+0000 7feca0ff9700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 0x7fec9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:55.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.563+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec94021070 con 0x7feca40d2320 2023-11-28T11:18:55.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.563+0000 7feca37fe700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 0x7fec9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:55.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.564+0000 7feca37fe700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 0x7fec9006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feca40b4d70 tx=0x7fec98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:55.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.567+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec94053d40 con 0x7feca40d2320 2023-11-28T11:18:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.768+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feca40745c0 con 0x7feca40d2320 2023-11-28T11:18:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.768+0000 7feca0ff9700 1 -- 172.21.15.53:0/334285252 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec94057360 con 0x7feca40d2320 2023-11-28T11:18:55.770 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.770+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 msgr2=0x7fec9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 0x7fec9006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feca40b4d70 tx=0x7fec98008040 comp rx=0 tx=0).stop 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 msgr2=0x7feca40c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec940090b0 tx=0x7fec94004750 comp rx=0 tx=0).stop 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 shutdown_connections 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fec900682f0 0x7fec9006a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feca4161340 0x7feca415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feca40d2320 0x7feca40c5f40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 --2- 172.21.15.53:0/334285252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feca415c9b0 0x7feca41f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:55.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 >> 172.21.15.53:0/334285252 conn(0x7feca4072600 msgr2=0x7feca406b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:55.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 shutdown_connections 2023-11-28T11:18:55.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.771+0000 7fecaa3ff700 1 -- 172.21.15.53:0/334285252 wait complete. 2023-11-28T11:18:55.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:18:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.996+0000 7f6dddbff700 1 Processor -- start 2023-11-28T11:18:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.996+0000 7f6dddbff700 1 -- start start 2023-11-28T11:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dddbff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dd80cd7c0 0x7f6dd80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dddbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd80d1950 con 0x7f6dd815c9b0 2023-11-28T11:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dddbff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd80ce140 con 0x7f6dd8161340 2023-11-28T11:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dddbff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd80b5800 con 0x7f6dd80cd7c0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dd77fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6dd815c9b0 0x7f6dd80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40030/0 (socket says 172.21.15.53:40030) 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dd77fe700 1 -- 172.21.15.53:0/3424733233 learned_addr learned my addr 172.21.15.53:0/3424733233 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.997+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/3424733233 >> v2:172.21.15.157:3300/0 conn(0x7f6dd80cd7c0 0x7f6dd80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104931443 0 0) 0x7f6dd80d1950 con 0x7f6dd815c9b0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc00035b0 con 0x7f6dd815c9b0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975599719 0 0) 0x7f6dd80ce140 con 0x7f6dd8161340 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd80d1950 con 0x7f6dd8161340 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361273230 0 0) 0x7f6dc00035b0 con 0x7f6dd815c9b0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 >> v2:172.21.15.157:3300/0 conn(0x7f6dd80cd7c0 msgr2=0x7f6dd80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 --2- 172.21.15.53:0/3424733233 >> v2:172.21.15.157:3300/0 conn(0x7f6dd80cd7c0 0x7f6dd80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 >> v1:172.21.15.157:6789/0 conn(0x7f6dd8161340 legacy=0x7f6dd80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd80d24f0 con 0x7f6dd815c9b0 2023-11-28T11:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f6dc8001670 con 0x7f6dd815c9b0 2023-11-28T11:18:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 >> v1:172.21.15.53:6789/0 conn(0x7f6dd815c9b0 legacy=0x7f6dd80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dc0003be0 0x7f6dc0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0004850 0x7f6dc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 0x7f6dc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80b5800 con 0x7f6dc0005250 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc000c160 con 0x7f6dc0004850 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc000c320 con 0x7f6dc0003be0 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.998+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 0x7f6dc000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd77fe700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dc0003be0 0x7f6dc0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd6ffd700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0004850 0x7f6dc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 -- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dc0003be0 msgr2=0x7f6dc0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dc0003be0 0x7f6dc0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 -- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0004850 msgr2=0x7f6dc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0004850 0x7f6dc0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 -- 172.21.15.53:0/3424733233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00035b0 con 0x7f6dc0005250 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd6ffd700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0004850 0x7f6dc0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 -- 172.21.15.53:0/3424733233 reap_dead start 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 0x7f6dc000ba10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009f90 tx=0x7f6dc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ac7ae1fabf820a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0003a20 con 0x7f6dc0005250 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd000f040 con 0x7f6dc0005250 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:55.999+0000 7f6dd67fc700 1 -- 172.21.15.53:0/3424733233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0003dc0 con 0x7f6dc0005250 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 -- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 msgr2=0x7f6dc000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 0x7f6dc000ba10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009f90 tx=0x7f6dc0005880 comp rx=0 tx=0).stop 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 -- 172.21.15.53:0/3424733233 shutdown_connections 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 --2- 172.21.15.53:0/3424733233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc0005250 0x7f6dc000ba10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 -- 172.21.15.53:0/3424733233 >> 172.21.15.53:0/3424733233 conn(0x7f6dd8072600 msgr2=0x7f6dd80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 -- 172.21.15.53:0/3424733233 shutdown_connections 2023-11-28T11:18:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.000+0000 7f6dddbff700 1 -- 172.21.15.53:0/3424733233 wait complete. 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.002+0000 7f6dddbff700 1 Processor -- start 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.002+0000 7f6dddbff700 1 -- start start 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0005250 0x7f6dd81e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 0x7f6dd815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80cc3e0 con 0x7f6dd8157540 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd81e6660 con 0x7f6dc0005250 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dddbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd81e6940 con 0x7f6dd80cbb00 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60718/0 (socket says 172.21.15.53:60718) 2023-11-28T11:18:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.003+0000 7f6dd6ffd700 1 -- 172.21.15.53:0/4097742886 learned_addr learned my addr 172.21.15.53:0/4097742886 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd77fe700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0005250 0x7f6dd81e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 0x7f6dd815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 -- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 msgr2=0x7f6dd815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 0x7f6dd815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 -- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0005250 msgr2=0x7f6dd81e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0005250 0x7f6dd81e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 -- 172.21.15.53:0/4097742886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd80d1300 con 0x7f6dd8157540 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd7fff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 0x7f6dd815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dd6ffd700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00e580 tx=0x7f6dcc00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0052e0 con 0x7f6dd8157540 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc005440 con 0x7f6dd8157540 2023-11-28T11:18:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0038c0 con 0x7f6dd8157540 2023-11-28T11:18:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.004+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd80b5520 con 0x7f6dd8157540 2023-11-28T11:18:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.005+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd80d15e0 con 0x7f6dd8157540 2023-11-28T11:18:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.006+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd81e64c0 con 0x7f6dd8157540 2023-11-28T11:18:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.006+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6dcc003a20 con 0x7f6dd8157540 2023-11-28T11:18:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.007+0000 7f6dbffff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 0x7f6dc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.007+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6dcc085090 con 0x7f6dd8157540 2023-11-28T11:18:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.008+0000 7f6dd77fe700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 0x7f6dc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.009+0000 7f6dd77fe700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 0x7f6dc406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dd815fb10 tx=0x7f6dc8000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.011+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6dd81e64c0 con 0x7f6dd8157540 2023-11-28T11:18:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.213+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dd81e64c0 con 0x7f6dd8157540 2023-11-28T11:18:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.213+0000 7f6dbffff700 1 -- 172.21.15.53:0/4097742886 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dd81e64c0 con 0x7f6dd8157540 2023-11-28T11:18:56.215 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 msgr2=0x7f6dc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 0x7f6dc406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dd815fb10 tx=0x7f6dc8000ed0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 msgr2=0x7f6dd8156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00e580 tx=0x7f6dcc00e940 comp rx=0 tx=0).stop 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 shutdown_connections 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6dc40682f0 0x7f6dc406a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dd80cbb00 0x7f6dd815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dc0005250 0x7f6dd81e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 --2- 172.21.15.53:0/4097742886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dd8157540 0x7f6dd8156e40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 >> 172.21.15.53:0/4097742886 conn(0x7f6dd8072600 msgr2=0x7f6dd80b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:56.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 shutdown_connections 2023-11-28T11:18:56.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.216+0000 7f6dddbff700 1 -- 172.21.15.53:0/4097742886 wait complete. 2023-11-28T11:18:56.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:18:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.435+0000 7f25195ff700 1 Processor -- start 2023-11-28T11:18:56.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.435+0000 7f25195ff700 1 -- start start 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25195ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 0x7f25140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25195ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140d1950 con 0x7f2514161340 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25195ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140d24f0 con 0x7f25140cd7c0 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25195ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b5800 con 0x7f251415c9b0 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25127fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2514161340 0x7f25140d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40034/0 (socket says 172.21.15.53:40034) 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f25127fc700 1 -- 172.21.15.53:0/1554447390 learned_addr learned my addr 172.21.15.53:0/1554447390 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f2512ffd700 1 --2- 172.21.15.53:0/1554447390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 0x7f25140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708911473 0 0) 0x7f25140d1950 con 0x7f2514161340 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.437+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24fc0035b0 con 0x7f2514161340 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836141348 0 0) 0x7f25140d24f0 con 0x7f25140cd7c0 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2512ffd700 1 -- 172.21.15.53:0/1554447390 >> v1:172.21.15.157:6789/0 conn(0x7f25140cd7c0 legacy=0x7f25140b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2512ffd700 1 -- 172.21.15.53:0/1554447390 >> v1:172.21.15.53:6789/0 conn(0x7f2514161340 legacy=0x7f25140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2512ffd700 1 -- 172.21.15.53:0/1554447390 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140b5520 con 0x7f251415c9b0 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2512ffd700 1 --2- 172.21.15.53:0/1554447390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 0x7f25140b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2504009870 tx=0x7f2504009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dddc2126c69bfe56 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011830 con 0x7f251415c9b0 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011990 con 0x7f251415c9b0 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.438+0000 7f2511ffb700 1 -- 172.21.15.53:0/1554447390 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011d00 con 0x7f251415c9b0 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 -- 172.21.15.53:0/1554447390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 msgr2=0x7f25140b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 --2- 172.21.15.53:0/1554447390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 0x7f25140b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2504009870 tx=0x7f2504009b80 comp rx=0 tx=0).stop 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 -- 172.21.15.53:0/1554447390 shutdown_connections 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 --2- 172.21.15.53:0/1554447390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251415c9b0 0x7f25140b4bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 -- 172.21.15.53:0/1554447390 >> 172.21.15.53:0/1554447390 conn(0x7f2514072600 msgr2=0x7f25140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 -- 172.21.15.53:0/1554447390 shutdown_connections 2023-11-28T11:18:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.439+0000 7f25195ff700 1 -- 172.21.15.53:0/1554447390 wait complete. 2023-11-28T11:18:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.441+0000 7f25195ff700 1 Processor -- start 2023-11-28T11:18:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.441+0000 7f25195ff700 1 -- start start 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25140cd7c0 0x7f2514156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2514161340 0x7f25141e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141e7de0 con 0x7f251415c9b0 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514157a60 con 0x7f25140cd7c0 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25195ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25141e71d0 con 0x7f2514161340 2023-11-28T11:18:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25127fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25127fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60734/0 (socket says 172.21.15.53:60734) 2023-11-28T11:18:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25127fc700 1 -- 172.21.15.53:0/3006746078 learned_addr learned my addr 172.21.15.53:0/3006746078 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f2512ffd700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25140cd7c0 0x7f2514156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.442+0000 7f25137fe700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2514161340 0x7f25141e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 -- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2514161340 msgr2=0x7f25141e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2514161340 0x7f25141e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 -- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25140cd7c0 msgr2=0x7f2514156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25140cd7c0 0x7f2514156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 -- 172.21.15.53:0/3006746078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040094a0 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25127fc700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f250800c420 tx=0x7f250800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508005d70 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250800ce30 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080072d0 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140d15e0 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.443+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140d1300 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.445+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25140b5130 con 0x7f251415c9b0 2023-11-28T11:18:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.445+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2508019030 con 0x7f251415c9b0 2023-11-28T11:18:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.446+0000 7f24fb7fe700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 0x7f250006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.446+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2508084a60 con 0x7f251415c9b0 2023-11-28T11:18:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.447+0000 7f2512ffd700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 0x7f250006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.447+0000 7f2512ffd700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 0x7f250006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25040090b0 tx=0x7f250401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.450+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f250804b5e0 con 0x7f251415c9b0 2023-11-28T11:18:56.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.656+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25140745c0 con 0x7f251415c9b0 2023-11-28T11:18:56.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.656+0000 7f24fb7fe700 1 -- 172.21.15.53:0/3006746078 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2508017020 con 0x7f251415c9b0 2023-11-28T11:18:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:56.657 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-28T11:18:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 2, 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "rbd" 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:18:56.664 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-28T11:18:56.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.658+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 msgr2=0x7f250006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.658+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 0x7f250006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25040090b0 tx=0x7f250401c040 comp rx=0 tx=0).stop 2023-11-28T11:18:56.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.658+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 msgr2=0x7f25140c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.658+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f250800c420 tx=0x7f250800c730 comp rx=0 tx=0).stop 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 shutdown_connections 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f25000682f0 0x7f250006a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2514161340 0x7f25141e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25140cd7c0 0x7f2514156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 --2- 172.21.15.53:0/3006746078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251415c9b0 0x7f25140c7520 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 >> 172.21.15.53:0/3006746078 conn(0x7f2514072600 msgr2=0x7f25140b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 shutdown_connections 2023-11-28T11:18:56.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.659+0000 7f25195ff700 1 -- 172.21.15.53:0/3006746078 wait complete. 2023-11-28T11:18:56.678 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:18:56.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:18:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.884+0000 7f30c8dff700 1 Processor -- start 2023-11-28T11:18:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.884+0000 7f30c8dff700 1 -- start start 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c8dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30c4161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c8dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c415e0a0 con 0x7f30c415dc30 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c8dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c4160d70 con 0x7f30c415a760 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c8dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415ab30 con 0x7f30c40cded0 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c2d9d700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f30c415dc30 0x7f30c40d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40050/0 (socket says 172.21.15.53:40050) 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.885+0000 7f30c2d9d700 1 -- 172.21.15.53:0/1883590112 learned_addr learned my addr 172.21.15.53:0/1883590112 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30c4161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960843103 0 0) 0x7f30c415e0a0 con 0x7f30c415dc30 2023-11-28T11:18:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ac0035b0 con 0x7f30c415dc30 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518014093 0 0) 0x7f30ac0035b0 con 0x7f30c415dc30 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 msgr2=0x7f30c4161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30c4161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 >> v1:172.21.15.157:6789/0 conn(0x7f30c415a760 legacy=0x7f30c40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40b5390 con 0x7f30c415dc30 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f30bc0019f0 con 0x7f30c415dc30 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 >> v1:172.21.15.53:6789/0 conn(0x7f30c415dc30 legacy=0x7f30c40d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30ac003f70 0x7f30ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac004d90 0x7f30ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415ab30 con 0x7f30ac004d90 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac00baf0 con 0x7f30ac003f70 2023-11-28T11:18:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.886+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac00bcb0 con 0x7f30c40cded0 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c2d9d700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac004d90 0x7f30ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30ac003f70 0x7f30ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c259c700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 -- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 msgr2=0x7f30ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40cded0 0x7f30ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 -- 172.21.15.53:0/1883590112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac004d90 msgr2=0x7f30ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac004d90 0x7f30ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c1d9b700 1 -- 172.21.15.53:0/1883590112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0035b0 con 0x7f30ac003f70 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.887+0000 7f30c2d9d700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac004d90 0x7f30ac005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30ac003f70 0x7f30ac004850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30c40ce2c0 tx=0x7f30ac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f69a3920340638 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800e6c0 con 0x7f30ac003f70 2023-11-28T11:18:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b800e820 con 0x7f30ac003f70 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c159a700 1 -- 172.21.15.53:0/1883590112 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800ebb0 con 0x7f30ac003f70 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c8dff700 1 -- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30ac003f70 msgr2=0x7f30ac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c8dff700 1 --2- 172.21.15.53:0/1883590112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30ac003f70 0x7f30ac004850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30c40ce2c0 tx=0x7f30ac005880 comp rx=0 tx=0).stop 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.888+0000 7f30c2d9d700 1 -- 172.21.15.53:0/1883590112 reap_dead start 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.889+0000 7f30c8dff700 1 -- 172.21.15.53:0/1883590112 shutdown_connections 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.889+0000 7f30c8dff700 1 -- 172.21.15.53:0/1883590112 >> 172.21.15.53:0/1883590112 conn(0x7f30c4072620 msgr2=0x7f30c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.889+0000 7f30c8dff700 1 -- 172.21.15.53:0/1883590112 shutdown_connections 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.889+0000 7f30c8dff700 1 -- 172.21.15.53:0/1883590112 wait complete. 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.890+0000 7f30c8dff700 1 Processor -- start 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.891+0000 7f30c8dff700 1 -- start start 2023-11-28T11:18:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.891+0000 7f30c8dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.891+0000 7f30c8dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40c9640 0x7f30c41485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c8dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 0x7f30c40cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c8dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40ccea0 con 0x7f30ac003f70 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c8dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40cc220 con 0x7f30c4148b30 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c8dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40cc500 con 0x7f30c40c9640 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60750/0 (socket says 172.21.15.53:60750) 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 -- 172.21.15.53:0/4187438667 learned_addr learned my addr 172.21.15.53:0/4187438667 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c2d9d700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 0x7f30c40cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40c9640 0x7f30c41485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 -- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40c9640 msgr2=0x7f30c41485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40c9640 0x7f30c41485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 -- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 msgr2=0x7f30c40cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 0x7f30c40cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 -- 172.21.15.53:0/4187438667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4160720 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c2d9d700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 0x7f30c40cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.892+0000 7f30c259c700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f30b400f770 tx=0x7f30b400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.893+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b40052e0 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.893+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4005460 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.893+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b40038c0 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.893+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c40b5670 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.893+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c4160a00 con 0x7f30ac003f70 2023-11-28T11:18:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.894+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c404e620 con 0x7f30ac003f70 2023-11-28T11:18:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.895+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30b4003a20 con 0x7f30ac003f70 2023-11-28T11:18:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.895+0000 7f30b2ffd700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 0x7f30a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.896+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f30b4085a00 con 0x7f30ac003f70 2023-11-28T11:18:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.896+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 0x7f30a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.897+0000 7f30c1d9b700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 0x7f30a406a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30b80090b0 tx=0x7f30b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:56.900+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f30b40524c0 con 0x7f30ac003f70 2023-11-28T11:18:57.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.102+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30c40d08a0 con 0x7f30ac003f70 2023-11-28T11:18:57.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.963+0000 7f30b2ffd700 1 -- 172.21.15.53:0/4187438667 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30b4003cb0 con 0x7f30ac003f70 2023-11-28T11:18:57.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.968+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 msgr2=0x7f30a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:57.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.968+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 0x7f30a406a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30b80090b0 tx=0x7f30b8008030 comp rx=0 tx=0).stop 2023-11-28T11:18:57.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.968+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 msgr2=0x7f30c40cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:57.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.968+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f30b400f770 tx=0x7f30b400fb30 comp rx=0 tx=0).stop 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 shutdown_connections 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f30a40682f0 0x7f30a406a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c4148b30 0x7f30c40cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ac003f70 0x7f30c40cbb80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 --2- 172.21.15.53:0/4187438667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c40c9640 0x7f30c41485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.969+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 >> 172.21.15.53:0/4187438667 conn(0x7f30c4072620 msgr2=0x7f30c40d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:57.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.970+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 shutdown_connections 2023-11-28T11:18:57.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:57.970+0000 7f30c8dff700 1 -- 172.21.15.53:0/4187438667 wait complete. 2023-11-28T11:18:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-11-28T11:18:57.994 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:18:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.205+0000 7f2d117ff700 1 Processor -- start 2023-11-28T11:18:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.205+0000 7f2d117ff700 1 -- start start 2023-11-28T11:18:58.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.206+0000 7f2d117ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d0c15c9b0 0x7f2d0c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d117ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0d1950 con 0x7f2d0c161340 2023-11-28T11:18:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d117ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0d24f0 con 0x7f2d0c0cd7c0 2023-11-28T11:18:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d117ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b5800 con 0x7f2d0c15c9b0 2023-11-28T11:18:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d03fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2d0c161340 0x7f2d0c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40052/0 (socket says 172.21.15.53:40052) 2023-11-28T11:18:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d03fff700 1 -- 172.21.15.53:0/2039172572 learned_addr learned my addr 172.21.15.53:0/2039172572 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:58.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d0affd700 1 --2- 172.21.15.53:0/2039172572 >> v2:172.21.15.157:3300/0 conn(0x7f2d0c15c9b0 0x7f2d0c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316590098 0 0) 0x7f2d0c0d1950 con 0x7f2d0c161340 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.207+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf80035b0 con 0x7f2d0c161340 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 27107512 0 0) 0x7f2cf80035b0 con 0x7f2d0c161340 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 >> v2:172.21.15.157:3300/0 conn(0x7f2d0c15c9b0 msgr2=0x7f2d0c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 --2- 172.21.15.53:0/2039172572 >> v2:172.21.15.157:3300/0 conn(0x7f2d0c15c9b0 0x7f2d0c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 >> v1:172.21.15.157:6789/0 conn(0x7f2d0c0cd7c0 legacy=0x7f2d0c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0b5520 con 0x7f2d0c161340 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f2cf4001670 con 0x7f2d0c161340 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 >> v1:172.21.15.53:6789/0 conn(0x7f2d0c161340 legacy=0x7f2d0c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2cf8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf8005200 0x7f2cf800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b5800 con 0x7f2cf8004850 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf800c110 con 0x7f2cf8003a00 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf800c2d0 con 0x7f2cf8005200 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2cf8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0affd700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.208+0000 7f2d0b7fe700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf8005200 0x7f2cf800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 -- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf8005200 msgr2=0x7f2cf800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cf8005200 0x7f2cf800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 -- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf8003a00 msgr2=0x7f2cf8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 -- 172.21.15.53:0/2039172572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80035b0 con 0x7f2cf8004850 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d0b7fe700 1 -- 172.21.15.53:0/2039172572 reap_dead start 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2cf8004cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c240 tx=0x7f2cf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cdd70dce8d0c7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007c10 con 0x7f2cf8004850 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf400ce00 con 0x7f2cf8004850 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d0a7fc700 1 -- 172.21.15.53:0/2039172572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4005ba0 con 0x7f2cf8004850 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.209+0000 7f2d117ff700 1 -- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 msgr2=0x7f2cf8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2cf8004cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c240 tx=0x7f2cf8005880 comp rx=0 tx=0).stop 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 -- 172.21.15.53:0/2039172572 shutdown_connections 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2039172572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2cf8004cc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 -- 172.21.15.53:0/2039172572 >> 172.21.15.53:0/2039172572 conn(0x7f2d0c072600 msgr2=0x7f2d0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 -- 172.21.15.53:0/2039172572 shutdown_connections 2023-11-28T11:18:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.210+0000 7f2d117ff700 1 -- 172.21.15.53:0/2039172572 wait complete. 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.211+0000 7f2d117ff700 1 Processor -- start 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.212+0000 7f2d117ff700 1 -- start start 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.212+0000 7f2d117ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.212+0000 7f2d117ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0c157a00 0x7f2d0c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.212+0000 7f2d117ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d0c0c8e40 0x7f2d0c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d117ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c9c90 con 0x7f2cf8004850 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d117ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c15eba0 con 0x7f2d0c0c8e40 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d117ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c92f0 con 0x7f2d0c157a00 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60764/0 (socket says 172.21.15.53:60764) 2023-11-28T11:18:58.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 -- 172.21.15.53:0/2025624180 learned_addr learned my addr 172.21.15.53:0/2025624180 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0b7fe700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d0c0c8e40 0x7f2d0c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0c157a00 0x7f2d0c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 -- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0c157a00 msgr2=0x7f2d0c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0c157a00 0x7f2d0c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 -- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d0c0c8e40 msgr2=0x7f2d0c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d0c0c8e40 0x7f2d0c156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 -- 172.21.15.53:0/2025624180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0d15e0 con 0x7f2cf8004850 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.213+0000 7f2d0affd700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011aa0 tx=0x7f2cfc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.214+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00f9e0 con 0x7f2cf8004850 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.214+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc00fb40 con 0x7f2cf8004850 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.214+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c0d1300 con 0x7f2cf8004850 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.214+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc014440 con 0x7f2cf8004850 2023-11-28T11:18:58.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.214+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c15cd80 con 0x7f2cf8004850 2023-11-28T11:18:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.215+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c0ca370 con 0x7f2cf8004850 2023-11-28T11:18:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.216+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2cfc01d400 con 0x7f2cf8004850 2023-11-28T11:18:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.217+0000 7f2d037fe700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.217+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2cfc086990 con 0x7f2cf8004850 2023-11-28T11:18:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.218+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:58.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.219+0000 7f2d03fff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 0x7f2cec06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4001930 tx=0x7f2cf4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:58.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.221+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2cfc053300 con 0x7f2cf8004850 2023-11-28T11:18:58.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.422+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2d0c0745c0 con 0x7f2cf8004850 2023-11-28T11:18:58.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.970+0000 7f2d037fe700 1 -- 172.21.15.53:0/2025624180 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2cfc056920 con 0x7f2cf8004850 2023-11-28T11:18:58.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.981+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 msgr2=0x7f2cec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.981+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 0x7f2cec06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4001930 tx=0x7f2cf4007000 comp rx=0 tx=0).stop 2023-11-28T11:18:58.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.981+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 msgr2=0x7f2d0c0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:58.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.981+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011aa0 tx=0x7f2cfc011e60 comp rx=0 tx=0).stop 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 shutdown_connections 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d0c0c8e40 0x7f2d0c156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf8004850 0x7f2d0c0c8900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 --2- 172.21.15.53:0/2025624180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0c157a00 0x7f2d0c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.985+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 >> 172.21.15.53:0/2025624180 conn(0x7f2d0c072600 msgr2=0x7f2d0c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:58.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.986+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 shutdown_connections 2023-11-28T11:18:58.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:58.986+0000 7f2d117ff700 1 -- 172.21.15.53:0/2025624180 wait complete. 2023-11-28T11:18:58.993 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-11-28T11:18:59.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:18:59.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.223+0000 7fc9cc7ff700 1 Processor -- start 2023-11-28T11:18:59.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.223+0000 7fc9cc7ff700 1 -- start start 2023-11-28T11:18:59.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cc7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9c4161340 0x7fc9c40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cc7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c40d1950 con 0x7fc9c40d2320 2023-11-28T11:18:59.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cc7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c40b5520 con 0x7fc9c415c9b0 2023-11-28T11:18:59.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cc7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c40b5800 con 0x7fc9c4161340 2023-11-28T11:18:59.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cad9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc9c40d2320 0x7fc9c40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40064/0 (socket says 172.21.15.53:40064) 2023-11-28T11:18:59.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9cad9c700 1 -- 172.21.15.53:0/2848690630 learned_addr learned my addr 172.21.15.53:0/2848690630 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:18:59.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.224+0000 7fc9c9d9a700 1 --2- 172.21.15.53:0/2848690630 >> v2:172.21.15.157:3300/0 conn(0x7fc9c4161340 0x7fc9c40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429796602 0 0) 0x7fc9c40d1950 con 0x7fc9c40d2320 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b00035b0 con 0x7fc9c40d2320 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2848690630 >> v1:172.21.15.157:6789/0 conn(0x7fc9c415c9b0 legacy=0x7fc9c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380378736 0 0) 0x7fc9c40b5520 con 0x7fc9c415c9b0 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2848690630 >> v1:172.21.15.53:6789/0 conn(0x7fc9c40d2320 legacy=0x7fc9c40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2848690630 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40d1300 con 0x7fc9c4161340 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.225+0000 7fc9c9d9a700 1 --2- 172.21.15.53:0/2848690630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c40d1de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009840 tx=0x7fc9b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45df13cc599e6b3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4011830 con 0x7fc9c4161340 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b4011990 con 0x7fc9c4161340 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9c9599700 1 -- 172.21.15.53:0/2848690630 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4011d20 con 0x7fc9c4161340 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2848690630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 msgr2=0x7fc9c40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2848690630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c40d1de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009840 tx=0x7fc9b4009b50 comp rx=0 tx=0).stop 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2848690630 shutdown_connections 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2848690630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c40d1de0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2848690630 >> 172.21.15.53:0/2848690630 conn(0x7fc9c4072600 msgr2=0x7fc9c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2848690630 shutdown_connections 2023-11-28T11:18:59.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.226+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2848690630 wait complete. 2023-11-28T11:18:59.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.228+0000 7fc9cc7ff700 1 Processor -- start 2023-11-28T11:18:59.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.228+0000 7fc9cc7ff700 1 -- start start 2023-11-28T11:18:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 0x7fc9c4151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c4154960 con 0x7fc9c415c9b0 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c4150860 con 0x7fc9c4161340 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9cc7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c4150d70 con 0x7fc9c40d2320 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9c9d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9c9d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60768/0 (socket says 172.21.15.53:60768) 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.229+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2251220475 learned_addr learned my addr 172.21.15.53:0/2251220475 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9cad9c700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9ca59b700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 0x7fc9c4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 msgr2=0x7fc9c4151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 0x7fc9c4151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 msgr2=0x7fc9c4153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c4153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 -- 172.21.15.53:0/2251220475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b40094a0 con 0x7fc9c415c9b0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9ca59b700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 0x7fc9c4151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9cad9c700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9c9d9a700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009b90 tx=0x7fc9b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b401ab20 con 0x7fc9c415c9b0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b401ac80 con 0x7fc9c415c9b0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c415cd80 con 0x7fc9c415c9b0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.230+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40243f0 con 0x7fc9c415c9b0 2023-11-28T11:18:59.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.231+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c40d15e0 con 0x7fc9c415c9b0 2023-11-28T11:18:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.232+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c415cf10 con 0x7fc9c415c9b0 2023-11-28T11:18:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.233+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9b400f050 con 0x7fc9c415c9b0 2023-11-28T11:18:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.234+0000 7fc9baffd700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 0x7fc9a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.234+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc9b40870b0 con 0x7fc9c415c9b0 2023-11-28T11:18:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.234+0000 7fc9ca59b700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 0x7fc9a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.235+0000 7fc9ca59b700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 0x7fc9a806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000f60 tx=0x7fc9c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.238+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9b4053880 con 0x7fc9c415c9b0 2023-11-28T11:18:59.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.488+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9c4074630 con 0x7fc9c415c9b0 2023-11-28T11:18:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.494+0000 7fc9baffd700 1 -- 172.21.15.53:0/2251220475 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc9c4074630 con 0x7fc9c415c9b0 2023-11-28T11:18:59.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.508+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 msgr2=0x7fc9a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.508+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 0x7fc9a806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000f60 tx=0x7fc9c0009040 comp rx=0 tx=0).stop 2023-11-28T11:18:59.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.508+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 msgr2=0x7fc9c4150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.508+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009b90 tx=0x7fc9b4004750 comp rx=0 tx=0).stop 2023-11-28T11:18:59.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 shutdown_connections 2023-11-28T11:18:59.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a80682f0 0x7fc9a806a7a0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000f60 tx=0x7fc9c0009040 comp rx=0 tx=0).stop 2023-11-28T11:18:59.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c40d2320 0x7fc9c4151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c4161340 0x7fc9c4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 --2- 172.21.15.53:0/2251220475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c415c9b0 0x7fc9c4150320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.509+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 >> 172.21.15.53:0/2251220475 conn(0x7fc9c4072600 msgr2=0x7fc9c415fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:59.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.512+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 shutdown_connections 2023-11-28T11:18:59.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.513+0000 7fc9cc7ff700 1 -- 172.21.15.53:0/2251220475 wait complete. 2023-11-28T11:18:59.520 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T11:18:59.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:18:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.744+0000 7f1faddff700 1 Processor -- start 2023-11-28T11:18:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.744+0000 7f1faddff700 1 -- start start 2023-11-28T11:18:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1faddff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fa8158ce0 0x7f1fa815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1faddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa806fb30 0x7f1fa80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1faddff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa806bf10 con 0x7f1fa80ce2e0 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1faddff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa80ce720 con 0x7f1fa8158ce0 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1faddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815d730 con 0x7f1fa806fb30 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1fa77fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fa8158ce0 0x7f1fa815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1fa77fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fa8158ce0 0x7f1fa815d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60778/0 (socket says 172.21.15.53:60778) 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1fa77fe700 1 -- 172.21.15.53:0/3684938524 learned_addr learned my addr 172.21.15.53:0/3684938524 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.745+0000 7f1f9ffff700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa806fb30 0x7f1fa80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa77fe700 1 -- 172.21.15.53:0/3684938524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa806fb30 msgr2=0x7f1fa80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa77fe700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa806fb30 0x7f1fa80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa77fe700 1 -- 172.21.15.53:0/3684938524 >> v1:172.21.15.157:6789/0 conn(0x7f1fa80ce2e0 legacy=0x7f1fa806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa77fe700 1 -- 172.21.15.53:0/3684938524 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa81590b0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa77fe700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa815d1f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f980098f0 tx=0x7f1f98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8da79193b21d8ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa6ffd700 1 -- 172.21.15.53:0/3684938524 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98011830 con 0x7f1fa8158ce0 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa6ffd700 1 -- 172.21.15.53:0/3684938524 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f98011990 con 0x7f1fa8158ce0 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.746+0000 7f1fa6ffd700 1 -- 172.21.15.53:0/3684938524 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98011d20 con 0x7f1fa8158ce0 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 -- 172.21.15.53:0/3684938524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 msgr2=0x7f1fa815d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa815d1f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f980098f0 tx=0x7f1f98009c00 comp rx=0 tx=0).stop 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 -- 172.21.15.53:0/3684938524 shutdown_connections 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa815d1f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 --2- 172.21.15.53:0/3684938524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa806fb30 0x7f1fa80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 -- 172.21.15.53:0/3684938524 >> 172.21.15.53:0/3684938524 conn(0x7f1fa80cb670 msgr2=0x7f1fa80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 -- 172.21.15.53:0/3684938524 shutdown_connections 2023-11-28T11:18:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.747+0000 7f1faddff700 1 -- 172.21.15.53:0/3684938524 wait complete. 2023-11-28T11:18:59.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.749+0000 7f1faddff700 1 Processor -- start 2023-11-28T11:18:59.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.749+0000 7f1faddff700 1 -- start start 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa806fb30 0x7f1fa80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa80ce2e0 0x7f1fa8157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80c5c30 con 0x7f1fa8158ce0 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80c6ee0 con 0x7f1fa806fb30 2023-11-28T11:18:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1faddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8158490 con 0x7f1fa80ce2e0 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60790/0 (socket says 172.21.15.53:60790) 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1fa7fff700 1 -- 172.21.15.53:0/3449341982 learned_addr learned my addr 172.21.15.53:0/3449341982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1f9ffff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa80ce2e0 0x7f1fa8157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.750+0000 7f1fa77fe700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa806fb30 0x7f1fa80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 -- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa80ce2e0 msgr2=0x7f1fa8157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa80ce2e0 0x7f1fa8157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 -- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa806fb30 msgr2=0x7f1fa80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa806fb30 0x7f1fa80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 -- 172.21.15.53:0/3449341982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009550 con 0x7f1fa8158ce0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa7fff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000cc80 tx=0x7f1fa000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0003c00 con 0x7f1fa8158ce0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0003d60 con 0x7f1fa8158ce0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000aaa0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa80cb3b0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.751+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa80cb0d0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.752+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa806b8c0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.753+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1fa0010050 con 0x7f1fa8158ce0 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.754+0000 7f1fa4ff9700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 0x7f1f8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.754+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fa0085bc0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.755+0000 7f1fa77fe700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 0x7f1f8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:18:59.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.755+0000 7f1fa77fe700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 0x7f1f8806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f98014040 tx=0x7f1f98008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:18:59.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.759+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1fa0052330 con 0x7f1fa8158ce0 2023-11-28T11:18:59.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.965+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fa80cbad0 con 0x7f1fa8158ce0 2023-11-28T11:18:59.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.966+0000 7f1fa4ff9700 1 -- 172.21.15.53:0/3449341982 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17992 (secure 0 0 0) 0x7f1fa0055950 con 0x7f1fa8158ce0 2023-11-28T11:18:59.968 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:18:59.968 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:59.490357+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 msgr2=0x7f1f8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 0x7f1f8806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f98014040 tx=0x7f1f98008030 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 msgr2=0x7f1fa80ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000cc80 tx=0x7f1fa000cf90 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 shutdown_connections 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1f880682f0 0x7f1f8806a7a0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f98014040 tx=0x7f1f98008030 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa8158ce0 0x7f1fa80ad970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa806fb30 0x7f1fa80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 --2- 172.21.15.53:0/3449341982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa80ce2e0 0x7f1fa8157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:18:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 >> 172.21.15.53:0/3449341982 conn(0x7f1fa80cb670 msgr2=0x7f1fa80ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:18:59.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 shutdown_connections 2023-11-28T11:18:59.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:18:59.969+0000 7f1faddff700 1 -- 172.21.15.53:0/3449341982 wait complete. 2023-11-28T11:18:59.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:00.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.189+0000 7fe8519ff700 1 Processor -- start 2023-11-28T11:19:00.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.189+0000 7fe8519ff700 1 -- start start 2023-11-28T11:19:00.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe8519ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c15c9b0 0x7fe84c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe8519ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe84c0cd7c0 0x7fe84c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe8519ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84c0ce160 con 0x7fe84c161340 2023-11-28T11:19:00.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe8519ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c0b5800 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe8519ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0d1950 con 0x7fe84c15c9b0 2023-11-28T11:19:00.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe843fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe84c161340 0x7fe84c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40072/0 (socket says 172.21.15.53:40072) 2023-11-28T11:19:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe843fff700 1 -- 172.21.15.53:0/2681624080 learned_addr learned my addr 172.21.15.53:0/2681624080 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe84b7fe700 1 --2- 172.21.15.53:0/2681624080 >> v2:172.21.15.53:3300/0 conn(0x7fe84c0cd7c0 0x7fe84c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.190+0000 7fe84affd700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c15c9b0 0x7fe84c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84a7fc700 1 -- 172.21.15.53:0/2681624080 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425555057 0 0) 0x7fe84c0ce160 con 0x7fe84c161340 2023-11-28T11:19:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84a7fc700 1 -- 172.21.15.53:0/2681624080 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8380035b0 con 0x7fe84c161340 2023-11-28T11:19:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84b7fe700 1 -- 172.21.15.53:0/2681624080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c15c9b0 msgr2=0x7fe84c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84b7fe700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c15c9b0 0x7fe84c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84b7fe700 1 -- 172.21.15.53:0/2681624080 >> v1:172.21.15.53:6789/0 conn(0x7fe84c161340 legacy=0x7fe84c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84b7fe700 1 -- 172.21.15.53:0/2681624080 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0d24f0 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84b7fe700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c0cdc20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe844009d90 tx=0x7fe84400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a3b0d798cfbec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84a7fc700 1 -- 172.21.15.53:0/2681624080 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400e070 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.191+0000 7fe84a7fc700 1 -- 172.21.15.53:0/2681624080 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400bb20 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe84a7fc700 1 -- 172.21.15.53:0/2681624080 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400be90 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 -- 172.21.15.53:0/2681624080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 msgr2=0x7fe84c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c0cdc20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe844009d90 tx=0x7fe84400b330 comp rx=0 tx=0).stop 2023-11-28T11:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 -- 172.21.15.53:0/2681624080 shutdown_connections 2023-11-28T11:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c0cdc20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 --2- 172.21.15.53:0/2681624080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c15c9b0 0x7fe84c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 -- 172.21.15.53:0/2681624080 >> 172.21.15.53:0/2681624080 conn(0x7fe84c072600 msgr2=0x7fe84c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 -- 172.21.15.53:0/2681624080 shutdown_connections 2023-11-28T11:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.192+0000 7fe8519ff700 1 -- 172.21.15.53:0/2681624080 wait complete. 2023-11-28T11:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.194+0000 7fe8519ff700 1 Processor -- start 2023-11-28T11:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.194+0000 7fe8519ff700 1 -- start start 2023-11-28T11:19:00.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 0x7fe84c145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c161340 0x7fe84c15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0c73b0 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c146740 con 0x7fe84c15c9b0 2023-11-28T11:19:00.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe8519ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c146a20 con 0x7fe84c161340 2023-11-28T11:19:00.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe84affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe84affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60812/0 (socket says 172.21.15.53:60812) 2023-11-28T11:19:00.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.195+0000 7fe84affd700 1 -- 172.21.15.53:0/4191755529 learned_addr learned my addr 172.21.15.53:0/4191755529 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:00.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 -- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c161340 msgr2=0x7fe84c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84b7fe700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c161340 0x7fe84c15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe843fff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 0x7fe84c145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c161340 0x7fe84c15a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 -- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 msgr2=0x7fe84c145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 0x7fe84c145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 -- 172.21.15.53:0/4191755529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009a70 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe843fff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 0x7fe84c145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe84affd700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00e870 tx=0x7fe83c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c009590 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c00ee70 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c0d1300 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00ca50 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.196+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c0b5520 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.197+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c0d15e0 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.198+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe83c019030 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.200+0000 7fe848ff9700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 0x7fe82c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.200+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe83c085c10 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.200+0000 7fe843fff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 0x7fe82c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.201+0000 7fe843fff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 0x7fe82c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe84c15b480 tx=0x7fe834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.204+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe83c052380 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.408+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe84c0c79e0 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.409+0000 7fe848ff9700 1 -- 172.21.15.53:0/4191755529 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17992 (secure 0 0 0) 0x7fe83c0559a0 con 0x7fe84c0cd7c0 2023-11-28T11:19:00.411 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:00.411 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:18:59.490357+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:00.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 msgr2=0x7fe82c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 0x7fe82c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe84c15b480 tx=0x7fe834009040 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 msgr2=0x7fe84c156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00e870 tx=0x7fe83c00ec30 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 shutdown_connections 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe82c0682f0 0x7fe82c06a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe84c161340 0x7fe84c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe84c0cd7c0 0x7fe84c156f40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 --2- 172.21.15.53:0/4191755529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c15c9b0 0x7fe84c145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 >> 172.21.15.53:0/4191755529 conn(0x7fe84c072600 msgr2=0x7fe84c0b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 shutdown_connections 2023-11-28T11:19:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.411+0000 7fe8519ff700 1 -- 172.21.15.53:0/4191755529 wait complete. 2023-11-28T11:19:00.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:19:00.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.637+0000 7fb0c6bff700 1 Processor -- start 2023-11-28T11:19:00.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.638+0000 7fb0c6bff700 1 -- start start 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c6bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c0161340 0x7fb0c00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c6bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb0c00d2320 0x7fb0c00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c6bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c00d1950 con 0x7fb0c015c9b0 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c6bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00b5800 con 0x7fb0c00d2320 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c6bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b5300 con 0x7fb0c0161340 2023-11-28T11:19:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c519c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb0c00d2320 0x7fb0c00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c499b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb0c015c9b0 0x7fb0c00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40080/0 (socket says 172.21.15.53:40080) 2023-11-28T11:19:00.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0c499b700 1 -- 172.21.15.53:0/2420457742 learned_addr learned my addr 172.21.15.53:0/2420457742 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:00.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.639+0000 7fb0bffff700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c0161340 0x7fb0c00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0c519c700 1 -- 172.21.15.53:0/2420457742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c0161340 msgr2=0x7fb0c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:00.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0c519c700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c0161340 0x7fb0c00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0c519c700 1 -- 172.21.15.53:0/2420457742 >> v1:172.21.15.53:6789/0 conn(0x7fb0c015c9b0 legacy=0x7fb0c00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0c519c700 1 -- 172.21.15.53:0/2420457742 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c00b5520 con 0x7fb0c00d2320 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0bf7fe700 1 -- 172.21.15.53:0/2420457742 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808813019 0 0) 0x7fb0c00d1950 con 0x7fb0c015c9b0 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0c519c700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c00d2320 0x7fb0c00cd7c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009dc0 tx=0x7fb0b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2839e9944344ed57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0bf7fe700 1 -- 172.21.15.53:0/2420457742 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800e070 con 0x7fb0c00d2320 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0bf7fe700 1 -- 172.21.15.53:0/2420457742 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800bb90 con 0x7fb0c00d2320 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.640+0000 7fb0bf7fe700 1 -- 172.21.15.53:0/2420457742 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8012430 con 0x7fb0c00d2320 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 -- 172.21.15.53:0/2420457742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c00d2320 msgr2=0x7fb0c00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c00d2320 0x7fb0c00cd7c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009dc0 tx=0x7fb0b800b330 comp rx=0 tx=0).stop 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 -- 172.21.15.53:0/2420457742 shutdown_connections 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c00d2320 0x7fb0c00cd7c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/2420457742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c0161340 0x7fb0c00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 -- 172.21.15.53:0/2420457742 >> 172.21.15.53:0/2420457742 conn(0x7fb0c0072600 msgr2=0x7fb0c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:00.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 -- 172.21.15.53:0/2420457742 shutdown_connections 2023-11-28T11:19:00.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.641+0000 7fb0c6bff700 1 -- 172.21.15.53:0/2420457742 wait complete. 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.643+0000 7fb0c6bff700 1 Processor -- start 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.643+0000 7fb0c6bff700 1 -- start start 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c00d2320 0x7fb0c01506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c0161340 0x7fb0c00c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c0150c20 con 0x7fb0c015c9b0 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c7c20 con 0x7fb0c0161340 2023-11-28T11:19:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c6bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c7010 con 0x7fb0c00d2320 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60822/0 (socket says 172.21.15.53:60822) 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 -- 172.21.15.53:0/1485444652 learned_addr learned my addr 172.21.15.53:0/1485444652 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c499b700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c00d2320 0x7fb0c01506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0c519c700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c0161340 0x7fb0c00c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 -- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c00d2320 msgr2=0x7fb0c01506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c00d2320 0x7fb0c01506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 -- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c0161340 msgr2=0x7fb0c00c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c0161340 0x7fb0c00c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.644+0000 7fb0bffff700 1 -- 172.21.15.53:0/1485444652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009a70 con 0x7fb0c015c9b0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0bffff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e810 tx=0x7fb0ac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac013610 con 0x7fb0c015c9b0 2023-11-28T11:19:00.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac013770 con 0x7fb0c015c9b0 2023-11-28T11:19:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c00d15e0 con 0x7fb0c015c9b0 2023-11-28T11:19:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac013b20 con 0x7fb0c015c9b0 2023-11-28T11:19:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.645+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c00d1300 con 0x7fb0c015c9b0 2023-11-28T11:19:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.646+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c015cd80 con 0x7fb0c015c9b0 2023-11-28T11:19:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.647+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0ac023070 con 0x7fb0c015c9b0 2023-11-28T11:19:00.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.648+0000 7fb0bd7fa700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 0x7fb0a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:00.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.648+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0ac085a70 con 0x7fb0c015c9b0 2023-11-28T11:19:00.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.649+0000 7fb0c499b700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 0x7fb0a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:00.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.649+0000 7fb0c499b700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 0x7fb0a406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000f60 tx=0x7fb0b4007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:00.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.652+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0ac052160 con 0x7fb0c015c9b0 2023-11-28T11:19:00.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:00.906+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb0c00745c0 con 0x7fb0c015c9b0 2023-11-28T11:19:00.940 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-11-28T11:19:00.938+0000 7f5c9b3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:19:01.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.509+0000 7fb0bd7fa700 1 -- 172.21.15.53:0/1485444652 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb0ac055780 con 0x7fb0c015c9b0 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 msgr2=0x7fb0a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 0x7fb0a406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000f60 tx=0x7fb0b4007c90 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 msgr2=0x7fb0c00b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e810 tx=0x7fb0ac00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 shutdown_connections 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb0a40682f0 0x7fb0a406a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c00d2320 0x7fb0c01506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c0161340 0x7fb0c00c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.511+0000 7fb0c6bff700 1 --2- 172.21.15.53:0/1485444652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0c015c9b0 0x7fb0c00b3840 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.512+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 >> 172.21.15.53:0/1485444652 conn(0x7fb0c0072600 msgr2=0x7fb0c00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:01.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.512+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 shutdown_connections 2023-11-28T11:19:01.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.512+0000 7fb0c6bff700 1 -- 172.21.15.53:0/1485444652 wait complete. 2023-11-28T11:19:01.533 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:19:01.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:19:01.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.739+0000 7f31857ff700 1 Processor -- start 2023-11-28T11:19:01.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.739+0000 7f31857ff700 1 -- start start 2023-11-28T11:19:01.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.740+0000 7f31857ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f318015c9b0 0x7f31800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.740+0000 7f31857ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 0x7f31800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f31857ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31800cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f31857ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31800b5800 con 0x7f318015c9b0 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f31857ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31800d1950 con 0x7f3180161340 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f31857ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31800b5520 con 0x7f31800d2320 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f318015c9b0 0x7f31800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f318015c9b0 0x7f31800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50506/0 (socket says 172.21.15.53:50506) 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 -- 172.21.15.53:0/1167294571 learned_addr learned my addr 172.21.15.53:0/1167294571 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f3177fff700 1 --2- 172.21.15.53:0/1167294571 >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 0x7f31800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317f7fe700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31800cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 -- 172.21.15.53:0/1167294571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 msgr2=0x7f31800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31800cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 -- 172.21.15.53:0/1167294571 >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 msgr2=0x7f31800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 --2- 172.21.15.53:0/1167294571 >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 0x7f31800d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f317effd700 1 -- 172.21.15.53:0/1167294571 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31800d1300 con 0x7f318015c9b0 2023-11-28T11:19:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.741+0000 7f3177fff700 1 --2- 172.21.15.53:0/1167294571 >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 0x7f31800d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f317effd700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f318015c9b0 0x7f31800b4bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31800685c0 tx=0x7f3170009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db15de37050bd21d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f317e7fc700 1 -- 172.21.15.53:0/1167294571 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000b870 con 0x7f318015c9b0 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f317e7fc700 1 -- 172.21.15.53:0/1167294571 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317000b9d0 con 0x7f318015c9b0 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f317e7fc700 1 -- 172.21.15.53:0/1167294571 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000bd60 con 0x7f318015c9b0 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f31857ff700 1 -- 172.21.15.53:0/1167294571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f318015c9b0 msgr2=0x7f31800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.742+0000 7f31857ff700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f318015c9b0 0x7f31800b4bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31800685c0 tx=0x7f3170009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 -- 172.21.15.53:0/1167294571 shutdown_connections 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31800cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 --2- 172.21.15.53:0/1167294571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f318015c9b0 0x7f31800b4bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 --2- 172.21.15.53:0/1167294571 >> v2:172.21.15.157:3300/0 conn(0x7f3180161340 0x7f31800d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 -- 172.21.15.53:0/1167294571 >> 172.21.15.53:0/1167294571 conn(0x7f3180072600 msgr2=0x7f31800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:01.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 -- 172.21.15.53:0/1167294571 shutdown_connections 2023-11-28T11:19:01.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.743+0000 7f31857ff700 1 -- 172.21.15.53:0/1167294571 wait complete. 2023-11-28T11:19:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.744+0000 7f31857ff700 1 Processor -- start 2023-11-28T11:19:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.745+0000 7f31857ff700 1 -- start start 2023-11-28T11:19:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31801515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f318015c9b0 0x7f3180150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3180150760 con 0x7f3180161340 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31801f3680 con 0x7f318015c9b0 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f31857ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31801f2a70 con 0x7f31800d2320 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317effd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31801515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50514/0 (socket says 172.21.15.53:50514) 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 -- 172.21.15.53:0/1053057739 learned_addr learned my addr 172.21.15.53:0/1053057739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f3177fff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f318015c9b0 0x7f3180150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 -- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 msgr2=0x7f31801515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31801515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 -- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f318015c9b0 msgr2=0x7f3180150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f318015c9b0 0x7f3180150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.746+0000 7f317f7fe700 1 -- 172.21.15.53:0/1053057739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3170009550 con 0x7f3180161340 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f317f7fe700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3178009fd0 tx=0x7f317800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178007ea0 con 0x7f3180161340 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317800ed30 con 0x7f3180161340 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317800eea0 con 0x7f3180161340 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31800b5130 con 0x7f3180161340 2023-11-28T11:19:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.747+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31800d15e0 con 0x7f3180161340 2023-11-28T11:19:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.748+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31800b52c0 con 0x7f3180161340 2023-11-28T11:19:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.749+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3178012050 con 0x7f3180161340 2023-11-28T11:19:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.750+0000 7f31777fe700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 0x7f316006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.750+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31780861d0 con 0x7f3180161340 2023-11-28T11:19:01.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.751+0000 7f317effd700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 0x7f316006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:01.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.752+0000 7f317effd700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 0x7f316006a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3170000c00 tx=0x7f3170008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:01.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:01.754+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3178052940 con 0x7f3180161340 2023-11-28T11:19:02.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.007+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f31800745c0 con 0x7f3180161340 2023-11-28T11:19:02.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.515+0000 7f31777fe700 1 -- 172.21.15.53:0/1053057739 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3178055f60 con 0x7f3180161340 2023-11-28T11:19:02.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 msgr2=0x7f316006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 0x7f316006a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3170000c00 tx=0x7f3170008030 comp rx=0 tx=0).stop 2023-11-28T11:19:02.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 msgr2=0x7f318015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3178009fd0 tx=0x7f317800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 shutdown_connections 2023-11-28T11:19:02.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f31600682f0 0x7f316006a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31800d2320 0x7f31801515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180161340 0x7f318015ec40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 --2- 172.21.15.53:0/1053057739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f318015c9b0 0x7f3180150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.517+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 >> 172.21.15.53:0/1053057739 conn(0x7f3180072600 msgr2=0x7f318015a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:02.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.518+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 shutdown_connections 2023-11-28T11:19:02.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.518+0000 7f31857ff700 1 -- 172.21.15.53:0/1053057739 wait complete. 2023-11-28T11:19:02.540 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:19:02.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:02.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.744+0000 7f41efdff700 1 Processor -- start 2023-11-28T11:19:02.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.744+0000 7f41efdff700 1 -- start start 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41efdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41efdff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e80d1950 con 0x7f41e80cd7c0 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41efdff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e80d24f0 con 0x7f41e8161340 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41efdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80b5800 con 0x7f41e815c9b0 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41ee39c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41e80cd7c0 0x7f41e80b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55132/0 (socket says 172.21.15.53:55132) 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41ee39c700 1 -- 172.21.15.53:0/891529968 learned_addr learned my addr 172.21.15.53:0/891529968 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.745+0000 7f41edb9b700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668594329 0 0) 0x7f41e80d1950 con 0x7f41e80cd7c0 2023-11-28T11:19:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d40035b0 con 0x7f41e80cd7c0 2023-11-28T11:19:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26287203 0 0) 0x7f41e80d24f0 con 0x7f41e8161340 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e80d1950 con 0x7f41e8161340 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1033769676 0 0) 0x7f41d40035b0 con 0x7f41e80cd7c0 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 msgr2=0x7f41e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41e80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 >> v1:172.21.15.157:6789/0 conn(0x7f41e8161340 legacy=0x7f41e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80b5520 con 0x7f41e80cd7c0 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f41e00019f0 con 0x7f41e80cd7c0 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 >> v1:172.21.15.53:6789/0 conn(0x7f41e80cd7c0 legacy=0x7f41e80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d4003bc0 0x7f41d4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d4004d90 0x7f41d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80b5800 con 0x7f41d4003bc0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d400bb10 con 0x7f41d4004d90 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d400bcd0 con 0x7f41e815c9b0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41edb9b700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d4003bc0 0x7f41d4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ee39c700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d4004d90 0x7f41d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.746+0000 7f41ed39a700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 -- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 msgr2=0x7f41d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 -- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d4004d90 msgr2=0x7f41d4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d4004d90 0x7f41d4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 -- 172.21.15.53:0/891529968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40035b0 con 0x7f41d4003bc0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41ed39a700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e815c9b0 0x7f41d4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41edb9b700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d4003bc0 0x7f41d4003fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009b60 tx=0x7f41d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a1525a45c99597 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400e6c0 con 0x7f41d4003bc0 2023-11-28T11:19:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e400e820 con 0x7f41d4003bc0 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.747+0000 7f41ecb99700 1 -- 172.21.15.53:0/891529968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400eb90 con 0x7f41d4003bc0 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 -- 172.21.15.53:0/891529968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d4003bc0 msgr2=0x7f41d4003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 --2- 172.21.15.53:0/891529968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d4003bc0 0x7f41d4003fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009b60 tx=0x7f41d4005880 comp rx=0 tx=0).stop 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41ee39c700 1 -- 172.21.15.53:0/891529968 reap_dead start 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 -- 172.21.15.53:0/891529968 shutdown_connections 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 -- 172.21.15.53:0/891529968 >> 172.21.15.53:0/891529968 conn(0x7f41e8072600 msgr2=0x7f41e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 -- 172.21.15.53:0/891529968 shutdown_connections 2023-11-28T11:19:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.748+0000 7f41efdff700 1 -- 172.21.15.53:0/891529968 wait complete. 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.750+0000 7f41efdff700 1 Processor -- start 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.750+0000 7f41efdff700 1 -- start start 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 0x7f41e8156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e80cb5b0 0x7f41e80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81f6060 con 0x7f41e8157880 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81f6340 con 0x7f41e80cb5b0 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41efdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81f6620 con 0x7f41d4003bc0 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41ee39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41ee39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50524/0 (socket says 172.21.15.53:50524) 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41ee39c700 1 -- 172.21.15.53:0/442289339 learned_addr learned my addr 172.21.15.53:0/442289339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41ed39a700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e80cb5b0 0x7f41e80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.751+0000 7f41edb9b700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 0x7f41e8156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 -- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 msgr2=0x7f41e8156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 0x7f41e8156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 -- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e80cb5b0 msgr2=0x7f41e80c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e80cb5b0 0x7f41e80c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 -- 172.21.15.53:0/442289339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80d15e0 con 0x7f41e8157880 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41edb9b700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 0x7f41e8156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41ee39c700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41e0000f80 tx=0x7f41e000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e000ac40 con 0x7f41e8157880 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e000adc0 con 0x7f41e8157880 2023-11-28T11:19:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e80d1300 con 0x7f41e8157880 2023-11-28T11:19:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0013630 con 0x7f41e8157880 2023-11-28T11:19:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.752+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e815cd80 con 0x7f41e8157880 2023-11-28T11:19:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.753+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e80745c0 con 0x7f41e8157880 2023-11-28T11:19:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.754+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f41e0023070 con 0x7f41e8157880 2023-11-28T11:19:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.755+0000 7f41de7fc700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 0x7f41cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.755+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41e0086660 con 0x7f41e8157880 2023-11-28T11:19:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.756+0000 7f41edb9b700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 0x7f41cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.756+0000 7f41edb9b700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 0x7f41cc06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41e40090b0 tx=0x7f41e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:02.759+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f41e0052d70 con 0x7f41e8157880 2023-11-28T11:19:03.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.010+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e80d08a0 con 0x7f41e8157880 2023-11-28T11:19:03.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.011+0000 7f41de7fc700 1 -- 172.21.15.53:0/442289339 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f41e0056390 con 0x7f41e8157880 2023-11-28T11:19:03.013 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:03.013 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:03.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 msgr2=0x7f41cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 0x7f41cc06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41e40090b0 tx=0x7f41e4008030 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 msgr2=0x7f41e80b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41e0000f80 tx=0x7f41e000a5b0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 shutdown_connections 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f41cc0682f0 0x7f41cc06a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41d4003bc0 0x7f41e8156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e8157880 0x7f41e80b7750 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 --2- 172.21.15.53:0/442289339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41e80cb5b0 0x7f41e80c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 >> 172.21.15.53:0/442289339 conn(0x7f41e8072600 msgr2=0x7f41e815f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 shutdown_connections 2023-11-28T11:19:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.013+0000 7f41efdff700 1 -- 172.21.15.53:0/442289339 wait complete. 2023-11-28T11:19:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:03.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.233+0000 7f25157ff700 1 Processor -- start 2023-11-28T11:19:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.233+0000 7f25157ff700 1 -- start start 2023-11-28T11:19:03.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.234+0000 7f25157ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f251015c9b0 0x7f25100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.234+0000 7f25157ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2510161340 0x7f25100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.234+0000 7f25157ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25100d1950 con 0x7f25100d2320 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.234+0000 7f25157ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25100b5800 con 0x7f251015c9b0 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.234+0000 7f25157ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25100b5520 con 0x7f2510161340 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250f7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f25100d2320 0x7f25100cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55144/0 (socket says 172.21.15.53:55144) 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250f7fe700 1 -- 172.21.15.53:0/2357237140 learned_addr learned my addr 172.21.15.53:0/2357237140 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 --2- 172.21.15.53:0/2357237140 >> v2:172.21.15.53:3300/0 conn(0x7f251015c9b0 0x7f25100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250e7fc700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2510161340 0x7f25100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250dffb700 1 -- 172.21.15.53:0/2357237140 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236582296 0 0) 0x7f25100d1950 con 0x7f25100d2320 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250dffb700 1 -- 172.21.15.53:0/2357237140 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f80035b0 con 0x7f25100d2320 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 -- 172.21.15.53:0/2357237140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2510161340 msgr2=0x7f25100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2510161340 0x7f25100d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 -- 172.21.15.53:0/2357237140 >> v1:172.21.15.53:6789/0 conn(0x7f25100d2320 legacy=0x7f25100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 -- 172.21.15.53:0/2357237140 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100d1300 con 0x7f251015c9b0 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.235+0000 7f250effd700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251015c9b0 0x7f25100b4bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2500009840 tx=0x7f2500009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b882ca74436aeda1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f250dffb700 1 -- 172.21.15.53:0/2357237140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500011830 con 0x7f251015c9b0 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f250dffb700 1 -- 172.21.15.53:0/2357237140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2500011990 con 0x7f251015c9b0 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f250dffb700 1 -- 172.21.15.53:0/2357237140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500011d20 con 0x7f251015c9b0 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 -- 172.21.15.53:0/2357237140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251015c9b0 msgr2=0x7f25100b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251015c9b0 0x7f25100b4bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2500009840 tx=0x7f2500009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 -- 172.21.15.53:0/2357237140 shutdown_connections 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251015c9b0 0x7f25100b4bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 --2- 172.21.15.53:0/2357237140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2510161340 0x7f25100d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 -- 172.21.15.53:0/2357237140 >> 172.21.15.53:0/2357237140 conn(0x7f2510072600 msgr2=0x7f25100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 -- 172.21.15.53:0/2357237140 shutdown_connections 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.236+0000 7f25157ff700 1 -- 172.21.15.53:0/2357237140 wait complete. 2023-11-28T11:19:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.239+0000 7f25157ff700 1 Processor -- start 2023-11-28T11:19:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.239+0000 7f25157ff700 1 -- start start 2023-11-28T11:19:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 0x7f25100c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 0x7f25100b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2510151930 con 0x7f2510161340 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100c67c0 con 0x7f25100d2320 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.240+0000 7f25157ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25101557b0 con 0x7f251015c9b0 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50542/0 (socket says 172.21.15.53:50542) 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 -- 172.21.15.53:0/25027235 learned_addr learned my addr 172.21.15.53:0/25027235 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250e7fc700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 0x7f25100b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 -- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 msgr2=0x7f25100b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250effd700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 0x7f25100c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 0x7f25100b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 -- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 msgr2=0x7f25100c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 0x7f25100c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 -- 172.21.15.53:0/25027235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000094a0 con 0x7f2510161340 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250effd700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 0x7f25100c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250e7fc700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 0x7f25100b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f250f7fe700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f250800cad0 tx=0x7f250800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800aac0 con 0x7f2510161340 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250800ac20 con 0x7f2510161340 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25100b5130 con 0x7f2510161340 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.241+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508012400 con 0x7f2510161340 2023-11-28T11:19:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.242+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25100d15e0 con 0x7f2510161340 2023-11-28T11:19:03.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.243+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25100b52c0 con 0x7f2510161340 2023-11-28T11:19:03.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.243+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2508019030 con 0x7f2510161340 2023-11-28T11:19:03.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.245+0000 7f24f77fe700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 0x7f24fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.245+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25080859b0 con 0x7f2510161340 2023-11-28T11:19:03.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.245+0000 7f250effd700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 0x7f24fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.246+0000 7f250effd700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 0x7f24fc06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f25000090b0 tx=0x7f2500008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.249+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2508052120 con 0x7f2510161340 2023-11-28T11:19:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.456+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25100745c0 con 0x7f2510161340 2023-11-28T11:19:03.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.457+0000 7f24f77fe700 1 -- 172.21.15.53:0/25027235 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f2508055740 con 0x7f2510161340 2023-11-28T11:19:03.460 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:03.460 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:00.492727+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 msgr2=0x7f24fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 0x7f24fc06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f25000090b0 tx=0x7f2500008030 comp rx=0 tx=0).stop 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 msgr2=0x7f25100b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f250800cad0 tx=0x7f250800ce90 comp rx=0 tx=0).stop 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 shutdown_connections 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f24fc0682f0 0x7f24fc06a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2510161340 0x7f25100b7660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25100d2320 0x7f25100c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 --2- 172.21.15.53:0/25027235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f251015c9b0 0x7f25100b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 >> 172.21.15.53:0/25027235 conn(0x7f2510072600 msgr2=0x7f251015cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:03.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 shutdown_connections 2023-11-28T11:19:03.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.460+0000 7f25157ff700 1 -- 172.21.15.53:0/25027235 wait complete. 2023-11-28T11:19:03.480 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:19:03.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:19:03.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.681+0000 7f4ab75ff700 1 Processor -- start 2023-11-28T11:19:03.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.682+0000 7f4ab75ff700 1 -- start start 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ab0161340 0x7f4ab00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00b5800 con 0x7f4ab015c9b0 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00d1950 con 0x7f4ab0161340 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab75ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b5520 con 0x7f4ab00d2320 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50544/0 (socket says 172.21.15.53:50544) 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 -- 172.21.15.53:0/1364935118 learned_addr learned my addr 172.21.15.53:0/1364935118 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab4b9a700 1 --2- 172.21.15.53:0/1364935118 >> v2:172.21.15.157:3300/0 conn(0x7f4ab0161340 0x7f4ab00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab5b9c700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 -- 172.21.15.53:0/1364935118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 msgr2=0x7f4ab00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 -- 172.21.15.53:0/1364935118 >> v2:172.21.15.157:3300/0 conn(0x7f4ab0161340 msgr2=0x7f4ab00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.683+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1364935118 >> v2:172.21.15.157:3300/0 conn(0x7f4ab0161340 0x7f4ab00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.684+0000 7f4ab539b700 1 -- 172.21.15.53:0/1364935118 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00d1300 con 0x7f4ab015c9b0 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.684+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00685c0 tx=0x7f4aac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651ee37889561a4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.684+0000 7f4aa7fff700 1 -- 172.21.15.53:0/1364935118 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00b870 con 0x7f4ab015c9b0 2023-11-28T11:19:03.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.684+0000 7f4aa7fff700 1 -- 172.21.15.53:0/1364935118 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac00b9d0 con 0x7f4ab015c9b0 2023-11-28T11:19:03.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.684+0000 7f4aa7fff700 1 -- 172.21.15.53:0/1364935118 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00bd60 con 0x7f4ab015c9b0 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1364935118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab015c9b0 msgr2=0x7f4ab00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00685c0 tx=0x7f4aac009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1364935118 shutdown_connections 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1364935118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab015c9b0 0x7f4ab00b4bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1364935118 >> v2:172.21.15.157:3300/0 conn(0x7f4ab0161340 0x7f4ab00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1364935118 >> 172.21.15.53:0/1364935118 conn(0x7f4ab0072600 msgr2=0x7f4ab00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1364935118 shutdown_connections 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.685+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1364935118 wait complete. 2023-11-28T11:19:03.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.686+0000 7f4ab75ff700 1 Processor -- start 2023-11-28T11:19:03.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.687+0000 7f4ab75ff700 1 -- start start 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.687+0000 7f4ab75ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab0150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.687+0000 7f4ab75ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab015c9b0 0x7f4ab015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab75ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab75ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01f32d0 con 0x7f4ab0161340 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab75ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01f3570 con 0x7f4ab015c9b0 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab75ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01f3850 con 0x7f4ab00d2320 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50546/0 (socket says 172.21.15.53:50546) 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 -- 172.21.15.53:0/1077428275 learned_addr learned my addr 172.21.15.53:0/1077428275 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab0150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab4b9a700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab015c9b0 0x7f4ab015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 -- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 msgr2=0x7f4ab0150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab0150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 -- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab015c9b0 msgr2=0x7f4ab015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab015c9b0 0x7f4ab015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.688+0000 7f4ab5b9c700 1 -- 172.21.15.53:0/1077428275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac009550 con 0x7f4ab0161340 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4ab5b9c700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009fd0 tx=0x7f4aa800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8007ea0 con 0x7f4ab0161340 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800ed30 con 0x7f4ab0161340 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab00b5130 con 0x7f4ab0161340 2023-11-28T11:19:03.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800eea0 con 0x7f4ab0161340 2023-11-28T11:19:03.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.689+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab00d15e0 con 0x7f4ab0161340 2023-11-28T11:19:03.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.690+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab00b52c0 con 0x7f4ab0161340 2023-11-28T11:19:03.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.691+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4aa8012050 con 0x7f4ab0161340 2023-11-28T11:19:03.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.692+0000 7f4aa5ffb700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:03.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.692+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4aa8086200 con 0x7f4ab0161340 2023-11-28T11:19:03.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.692+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:03.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.693+0000 7f4ab539b700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 0x7f4a9406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4aac000c00 tx=0x7f4aac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:03.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.696+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4aa8052970 con 0x7f4ab0161340 2023-11-28T11:19:03.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.894+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ab00745c0 con 0x7f4a940682f0 2023-11-28T11:19:03.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.906+0000 7f4aa5ffb700 1 -- 172.21.15.53:0/1077428275 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310843 (secure 0 0 0) 0x7f4ab00745c0 con 0x7f4a940682f0 2023-11-28T11:19:03.913 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:03.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 msgr2=0x7f4a9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 0x7f4a9406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4aac000c00 tx=0x7f4aac008030 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 msgr2=0x7f4ab015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009fd0 tx=0x7f4aa800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 shutdown_connections 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab00d2320 0x7f4ab0150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ab0161340 0x7f4ab015f160 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 --2- 172.21.15.53:0/1077428275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab015c9b0 0x7f4ab015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 >> 172.21.15.53:0/1077428275 conn(0x7f4ab0072600 msgr2=0x7f4ab015a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 shutdown_connections 2023-11-28T11:19:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:03.910+0000 7f4ab75ff700 1 -- 172.21.15.53:0/1077428275 wait complete. 2023-11-28T11:19:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:19:03.931 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-11-28T11:19:02.948220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":102576,"kb_used_data":4336,"kb_used_omap":0,"kb_used_meta":98176,"kb_avail":754872144,"statfs":{"total":773094113280,"available":772989075456,"internally_reserved":0,"allocated":4440064,"data_stored":2505353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":100532224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985850"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517971+0000","last_change":"2023-11-28T11:18:59.513740+0000","last_active":"2023-11-28T11:18:59.517971+0000","last_peered":"2023-11-28T11:18:59.517971+0000","last_clean":"2023-11-28T11:18:59.517971+0000","last_became_active":"2023-11-28T11:18:59.513004+0000","last_became_peered":"2023-11-28T11:18:59.513004+0000","last_unstale":"2023-11-28T11:18:59.517971+0000","last_undegraded":"2023-11-28T11:18:59.517971+0000","last_fullsized":"2023-11-28T11:18:59.517971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:09.605544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519842+0000","last_change":"2023-11-28T11:18:59.518148+0000","last_active":"2023-11-28T11:18:59.519842+0000","last_peered":"2023-11-28T11:18:59.519842+0000","last_clean":"2023-11-28T11:18:59.519842+0000","last_became_active":"2023-11-28T11:18:59.517868+0000","last_became_peered":"2023-11-28T11:18:59.517868+0000","last_unstale":"2023-11-28T11:18:59.519842+0000","last_undegraded":"2023-11-28T11:18:59.519842+0000","last_fullsized":"2023-11-28T11:18:59.519842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532776+0000","last_change":"2023-11-28T11:18:59.530889+0000","last_active":"2023-11-28T11:18:59.532776+0000","last_peered":"2023-11-28T11:18:59.532776+0000","last_clean":"2023-11-28T11:18:59.532776+0000","last_became_active":"2023-11-28T11:18:59.530572+0000","last_became_peered":"2023-11-28T11:18:59.530572+0000","last_unstale":"2023-11-28T11:18:59.532776+0000","last_undegraded":"2023-11-28T11:18:59.532776+0000","last_fullsized":"2023-11-28T11:18:59.532776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.528454+0000","last_change":"2023-11-28T11:18:59.527366+0000","last_active":"2023-11-28T11:18:59.528454+0000","last_peered":"2023-11-28T11:18:59.528454+0000","last_clean":"2023-11-28T11:18:59.528454+0000","last_became_active":"2023-11-28T11:18:59.527133+0000","last_became_peered":"2023-11-28T11:18:59.527133+0000","last_unstale":"2023-11-28T11:18:59.528454+0000","last_undegraded":"2023-11-28T11:18:59.528454+0000","last_fullsized":"2023-11-28T11:18:59.528454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:04.340204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522343+0000","last_change":"2023-11-28T11:18:59.520216+0000","last_active":"2023-11-28T11:18:59.522343+0000","last_peered":"2023-11-28T11:18:59.522343+0000","last_clean":"2023-11-28T11:18:59.522343+0000","last_became_active":"2023-11-28T11:18:59.519741+0000","last_became_peered":"2023-11-28T11:18:59.519741+0000","last_unstale":"2023-11-28T11:18:59.522343+0000","last_undegraded":"2023-11-28T11:18:59.522343+0000","last_fullsized":"2023-11-28T11:18:59.522343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:28.121736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.496887+0000","last_change":"2023-11-28T11:18:59.516703+0000","last_active":"2023-11-28T11:19:00.496887+0000","last_peered":"2023-11-28T11:19:00.496887+0000","last_clean":"2023-11-28T11:19:00.496887+0000","last_became_active":"2023-11-28T11:18:59.516422+0000","last_became_peered":"2023-11-28T11:18:59.516422+0000","last_unstale":"2023-11-28T11:19:00.496887+0000","last_undegraded":"2023-11-28T11:19:00.496887+0000","last_fullsized":"2023-11-28T11:19:00.496887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:54.590713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498567+0000","last_change":"2023-11-28T11:18:59.526139+0000","last_active":"2023-11-28T11:19:00.498567+0000","last_peered":"2023-11-28T11:19:00.498567+0000","last_clean":"2023-11-28T11:19:00.498567+0000","last_became_active":"2023-11-28T11:18:59.525351+0000","last_became_peered":"2023-11-28T11:18:59.525351+0000","last_unstale":"2023-11-28T11:19:00.498567+0000","last_undegraded":"2023-11-28T11:19:00.498567+0000","last_fullsized":"2023-11-28T11:19:00.498567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:29.556331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532936+0000","last_change":"2023-11-28T11:18:59.531153+0000","last_active":"2023-11-28T11:18:59.532936+0000","last_peered":"2023-11-28T11:18:59.532936+0000","last_clean":"2023-11-28T11:18:59.532936+0000","last_became_active":"2023-11-28T11:18:59.528513+0000","last_became_peered":"2023-11-28T11:18:59.528513+0000","last_unstale":"2023-11-28T11:18:59.532936+0000","last_undegraded":"2023-11-28T11:18:59.532936+0000","last_fullsized":"2023-11-28T11:18:59.532936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:53.259503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532719+0000","last_change":"2023-11-28T11:18:59.530510+0000","last_active":"2023-11-28T11:18:59.532719+0000","last_peered":"2023-11-28T11:18:59.532719+0000","last_clean":"2023-11-28T11:18:59.532719+0000","last_became_active":"2023-11-28T11:18:59.530179+0000","last_became_peered":"2023-11-28T11:18:59.530179+0000","last_unstale":"2023-11-28T11:18:59.532719+0000","last_undegraded":"2023-11-28T11:18:59.532719+0000","last_fullsized":"2023-11-28T11:18:59.532719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498958+0000","last_change":"2023-11-28T11:18:59.527077+0000","last_active":"2023-11-28T11:19:00.498958+0000","last_peered":"2023-11-28T11:19:00.498958+0000","last_clean":"2023-11-28T11:19:00.498958+0000","last_became_active":"2023-11-28T11:18:59.526784+0000","last_became_peered":"2023-11-28T11:18:59.526784+0000","last_unstale":"2023-11-28T11:19:00.498958+0000","last_undegraded":"2023-11-28T11:19:00.498958+0000","last_fullsized":"2023-11-28T11:19:00.498958+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:41:04.591178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499193+0000","last_change":"2023-11-28T11:18:59.522768+0000","last_active":"2023-11-28T11:19:00.499193+0000","last_peered":"2023-11-28T11:19:00.499193+0000","last_clean":"2023-11-28T11:19:00.499193+0000","last_became_active":"2023-11-28T11:18:59.522477+0000","last_became_peered":"2023-11-28T11:18:59.522477+0000","last_unstale":"2023-11-28T11:19:00.499193+0000","last_undegraded":"2023-11-28T11:19:00.499193+0000","last_fullsized":"2023-11-28T11:19:00.499193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:28:21.182281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522219+0000","last_change":"2023-11-28T11:18:59.520364+0000","last_active":"2023-11-28T11:18:59.522219+0000","last_peered":"2023-11-28T11:18:59.522219+0000","last_clean":"2023-11-28T11:18:59.522219+0000","last_became_active":"2023-11-28T11:18:59.520093+0000","last_became_peered":"2023-11-28T11:18:59.520093+0000","last_unstale":"2023-11-28T11:18:59.522219+0000","last_undegraded":"2023-11-28T11:18:59.522219+0000","last_fullsized":"2023-11-28T11:18:59.522219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497993+0000","last_change":"2023-11-28T11:18:59.526339+0000","last_active":"2023-11-28T11:19:00.497993+0000","last_peered":"2023-11-28T11:19:00.497993+0000","last_clean":"2023-11-28T11:19:00.497993+0000","last_became_active":"2023-11-28T11:18:59.525957+0000","last_became_peered":"2023-11-28T11:18:59.525957+0000","last_unstale":"2023-11-28T11:19:00.497993+0000","last_undegraded":"2023-11-28T11:19:00.497993+0000","last_fullsized":"2023-11-28T11:19:00.497993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:37.757891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498325+0000","last_change":"2023-11-28T11:18:59.523717+0000","last_active":"2023-11-28T11:19:00.498325+0000","last_peered":"2023-11-28T11:19:00.498325+0000","last_clean":"2023-11-28T11:19:00.498325+0000","last_became_active":"2023-11-28T11:18:59.523454+0000","last_became_peered":"2023-11-28T11:18:59.523454+0000","last_unstale":"2023-11-28T11:19:00.498325+0000","last_undegraded":"2023-11-28T11:19:00.498325+0000","last_fullsized":"2023-11-28T11:19:00.498325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:18.743726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498382+0000","last_change":"2023-11-28T11:18:59.525698+0000","last_active":"2023-11-28T11:19:00.498382+0000","last_peered":"2023-11-28T11:19:00.498382+0000","last_clean":"2023-11-28T11:19:00.498382+0000","last_became_active":"2023-11-28T11:18:59.525004+0000","last_became_peered":"2023-11-28T11:18:59.525004+0000","last_unstale":"2023-11-28T11:19:00.498382+0000","last_undegraded":"2023-11-28T11:19:00.498382+0000","last_fullsized":"2023-11-28T11:19:00.498382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:27.462121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532313+0000","last_change":"2023-11-28T11:18:59.530295+0000","last_active":"2023-11-28T11:18:59.532313+0000","last_peered":"2023-11-28T11:18:59.532313+0000","last_clean":"2023-11-28T11:18:59.532313+0000","last_became_active":"2023-11-28T11:18:59.530001+0000","last_became_peered":"2023-11-28T11:18:59.530001+0000","last_unstale":"2023-11-28T11:18:59.532313+0000","last_undegraded":"2023-11-28T11:18:59.532313+0000","last_fullsized":"2023-11-28T11:18:59.532313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532745+0000","last_change":"2023-11-28T11:18:59.530935+0000","last_active":"2023-11-28T11:18:59.532745+0000","last_peered":"2023-11-28T11:18:59.532745+0000","last_clean":"2023-11-28T11:18:59.532745+0000","last_became_active":"2023-11-28T11:18:59.530436+0000","last_became_peered":"2023-11-28T11:18:59.530436+0000","last_unstale":"2023-11-28T11:18:59.532745+0000","last_undegraded":"2023-11-28T11:18:59.532745+0000","last_fullsized":"2023-11-28T11:18:59.532745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747752+0000","last_change":"2023-11-28T11:18:59.520394+0000","last_active":"2023-11-28T11:19:00.747752+0000","last_peered":"2023-11-28T11:19:00.747752+0000","last_clean":"2023-11-28T11:19:00.747752+0000","last_became_active":"2023-11-28T11:18:59.519198+0000","last_became_peered":"2023-11-28T11:18:59.519198+0000","last_unstale":"2023-11-28T11:19:00.747752+0000","last_undegraded":"2023-11-28T11:19:00.747752+0000","last_fullsized":"2023-11-28T11:19:00.747752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:53.850423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519945+0000","last_change":"2023-11-28T11:18:59.514188+0000","last_active":"2023-11-28T11:18:59.519945+0000","last_peered":"2023-11-28T11:18:59.519945+0000","last_clean":"2023-11-28T11:18:59.519945+0000","last_became_active":"2023-11-28T11:18:59.513929+0000","last_became_peered":"2023-11-28T11:18:59.513929+0000","last_unstale":"2023-11-28T11:18:59.519945+0000","last_undegraded":"2023-11-28T11:18:59.519945+0000","last_fullsized":"2023-11-28T11:18:59.519945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:35.053943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747636+0000","last_change":"2023-11-28T11:18:59.530152+0000","last_active":"2023-11-28T11:19:00.747636+0000","last_peered":"2023-11-28T11:19:00.747636+0000","last_clean":"2023-11-28T11:19:00.747636+0000","last_became_active":"2023-11-28T11:18:59.528030+0000","last_became_peered":"2023-11-28T11:18:59.528030+0000","last_unstale":"2023-11-28T11:19:00.747636+0000","last_undegraded":"2023-11-28T11:19:00.747636+0000","last_fullsized":"2023-11-28T11:19:00.747636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:16.864464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498030+0000","last_change":"2023-11-28T11:18:59.521195+0000","last_active":"2023-11-28T11:19:00.498030+0000","last_peered":"2023-11-28T11:19:00.498030+0000","last_clean":"2023-11-28T11:19:00.498030+0000","last_became_active":"2023-11-28T11:18:59.515301+0000","last_became_peered":"2023-11-28T11:18:59.515301+0000","last_unstale":"2023-11-28T11:19:00.498030+0000","last_undegraded":"2023-11-28T11:19:00.498030+0000","last_fullsized":"2023-11-28T11:19:00.498030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:56:01.821322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498496+0000","last_change":"2023-11-28T11:18:59.515641+0000","last_active":"2023-11-28T11:19:00.498496+0000","last_peered":"2023-11-28T11:19:00.498496+0000","last_clean":"2023-11-28T11:19:00.498496+0000","last_became_active":"2023-11-28T11:18:59.515343+0000","last_became_peered":"2023-11-28T11:18:59.515343+0000","last_unstale":"2023-11-28T11:19:00.498496+0000","last_undegraded":"2023-11-28T11:19:00.498496+0000","last_fullsized":"2023-11-28T11:19:00.498496+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:04.494294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525309+0000","last_change":"2023-11-28T11:18:59.520502+0000","last_active":"2023-11-28T11:18:59.525309+0000","last_peered":"2023-11-28T11:18:59.525309+0000","last_clean":"2023-11-28T11:18:59.525309+0000","last_became_active":"2023-11-28T11:18:59.519898+0000","last_became_peered":"2023-11-28T11:18:59.519898+0000","last_unstale":"2023-11-28T11:18:59.525309+0000","last_undegraded":"2023-11-28T11:18:59.525309+0000","last_fullsized":"2023-11-28T11:18:59.525309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525436+0000","last_change":"2023-11-28T11:18:59.524078+0000","last_active":"2023-11-28T11:18:59.525436+0000","last_peered":"2023-11-28T11:18:59.525436+0000","last_clean":"2023-11-28T11:18:59.525436+0000","last_became_active":"2023-11-28T11:18:59.523785+0000","last_became_peered":"2023-11-28T11:18:59.523785+0000","last_unstale":"2023-11-28T11:18:59.525436+0000","last_undegraded":"2023-11-28T11:18:59.525436+0000","last_fullsized":"2023-11-28T11:18:59.525436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:38.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747802+0000","last_change":"2023-11-28T11:18:59.520009+0000","last_active":"2023-11-28T11:19:00.747802+0000","last_peered":"2023-11-28T11:19:00.747802+0000","last_clean":"2023-11-28T11:19:00.747802+0000","last_became_active":"2023-11-28T11:18:59.517949+0000","last_became_peered":"2023-11-28T11:18:59.517949+0000","last_unstale":"2023-11-28T11:19:00.747802+0000","last_undegraded":"2023-11-28T11:19:00.747802+0000","last_fullsized":"2023-11-28T11:19:00.747802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:01.731030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497235+0000","last_change":"2023-11-28T11:18:59.528917+0000","last_active":"2023-11-28T11:19:00.497235+0000","last_peered":"2023-11-28T11:19:00.497235+0000","last_clean":"2023-11-28T11:19:00.497235+0000","last_became_active":"2023-11-28T11:18:59.528647+0000","last_became_peered":"2023-11-28T11:18:59.528647+0000","last_unstale":"2023-11-28T11:19:00.497235+0000","last_undegraded":"2023-11-28T11:19:00.497235+0000","last_fullsized":"2023-11-28T11:19:00.497235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.468149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747627+0000","last_change":"2023-11-28T11:18:59.520290+0000","last_active":"2023-11-28T11:19:00.747627+0000","last_peered":"2023-11-28T11:19:00.747627+0000","last_clean":"2023-11-28T11:19:00.747627+0000","last_became_active":"2023-11-28T11:18:59.519328+0000","last_became_peered":"2023-11-28T11:18:59.519328+0000","last_unstale":"2023-11-28T11:19:00.747627+0000","last_undegraded":"2023-11-28T11:19:00.747627+0000","last_fullsized":"2023-11-28T11:19:00.747627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:27.729983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498935+0000","last_change":"2023-11-28T11:18:59.527402+0000","last_active":"2023-11-28T11:19:00.498935+0000","last_peered":"2023-11-28T11:19:00.498935+0000","last_clean":"2023-11-28T11:19:00.498935+0000","last_became_active":"2023-11-28T11:18:59.525202+0000","last_became_peered":"2023-11-28T11:18:59.525202+0000","last_unstale":"2023-11-28T11:19:00.498935+0000","last_undegraded":"2023-11-28T11:19:00.498935+0000","last_fullsized":"2023-11-28T11:19:00.498935+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:38.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747591+0000","last_change":"2023-11-28T11:18:59.513537+0000","last_active":"2023-11-28T11:19:00.747591+0000","last_peered":"2023-11-28T11:19:00.747591+0000","last_clean":"2023-11-28T11:19:00.747591+0000","last_became_active":"2023-11-28T11:18:59.513253+0000","last_became_peered":"2023-11-28T11:18:59.513253+0000","last_unstale":"2023-11-28T11:19:00.747591+0000","last_undegraded":"2023-11-28T11:19:00.747591+0000","last_fullsized":"2023-11-28T11:19:00.747591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:32.314154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747503+0000","last_change":"2023-11-28T11:18:59.520300+0000","last_active":"2023-11-28T11:19:00.747503+0000","last_peered":"2023-11-28T11:19:00.747503+0000","last_clean":"2023-11-28T11:19:00.747503+0000","last_became_active":"2023-11-28T11:18:59.519955+0000","last_became_peered":"2023-11-28T11:18:59.519955+0000","last_unstale":"2023-11-28T11:19:00.747503+0000","last_undegraded":"2023-11-28T11:19:00.747503+0000","last_fullsized":"2023-11-28T11:19:00.747503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:30.358902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525436+0000","last_change":"2023-11-28T11:18:59.520674+0000","last_active":"2023-11-28T11:18:59.525436+0000","last_peered":"2023-11-28T11:18:59.525436+0000","last_clean":"2023-11-28T11:18:59.525436+0000","last_became_active":"2023-11-28T11:18:59.520251+0000","last_became_peered":"2023-11-28T11:18:59.520251+0000","last_unstale":"2023-11-28T11:18:59.525436+0000","last_undegraded":"2023-11-28T11:18:59.525436+0000","last_fullsized":"2023-11-28T11:18:59.525436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497284+0000","last_change":"2023-11-28T11:18:59.518778+0000","last_active":"2023-11-28T11:19:00.497284+0000","last_peered":"2023-11-28T11:19:00.497284+0000","last_clean":"2023-11-28T11:19:00.497284+0000","last_became_active":"2023-11-28T11:18:59.518535+0000","last_became_peered":"2023-11-28T11:18:59.518535+0000","last_unstale":"2023-11-28T11:19:00.497284+0000","last_undegraded":"2023-11-28T11:19:00.497284+0000","last_fullsized":"2023-11-28T11:19:00.497284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:28.302627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.531686+0000","last_change":"2023-11-28T11:18:59.529081+0000","last_active":"2023-11-28T11:18:59.531686+0000","last_peered":"2023-11-28T11:18:59.531686+0000","last_clean":"2023-11-28T11:18:59.531686+0000","last_became_active":"2023-11-28T11:18:59.528801+0000","last_became_peered":"2023-11-28T11:18:59.528801+0000","last_unstale":"2023-11-28T11:18:59.531686+0000","last_undegraded":"2023-11-28T11:18:59.531686+0000","last_fullsized":"2023-11-28T11:18:59.531686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747861+0000","last_change":"2023-11-28T11:18:58.982271+0000","last_active":"2023-11-28T11:19:00.747861+0000","last_peered":"2023-11-28T11:19:00.747861+0000","last_clean":"2023-11-28T11:19:00.747861+0000","last_became_active":"2023-11-28T11:18:58.981951+0000","last_became_peered":"2023-11-28T11:18:58.981951+0000","last_unstale":"2023-11-28T11:19:00.747861+0000","last_undegraded":"2023-11-28T11:19:00.747861+0000","last_fullsized":"2023-11-28T11:19:00.747861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:35.247644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497386+0000","last_change":"2023-11-28T11:18:59.531050+0000","last_active":"2023-11-28T11:19:00.497386+0000","last_peered":"2023-11-28T11:19:00.497386+0000","last_clean":"2023-11-28T11:19:00.497386+0000","last_became_active":"2023-11-28T11:18:59.529001+0000","last_became_peered":"2023-11-28T11:18:59.529001+0000","last_unstale":"2023-11-28T11:19:00.497386+0000","last_undegraded":"2023-11-28T11:19:00.497386+0000","last_fullsized":"2023-11-28T11:19:00.497386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:53.259503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524539+0000","last_change":"2023-11-28T11:18:58.982012+0000","last_active":"2023-11-28T11:18:59.524539+0000","last_peered":"2023-11-28T11:18:59.524539+0000","last_clean":"2023-11-28T11:18:59.524539+0000","last_became_active":"2023-11-28T11:18:58.980911+0000","last_became_peered":"2023-11-28T11:18:58.980911+0000","last_unstale":"2023-11-28T11:18:59.524539+0000","last_undegraded":"2023-11-28T11:18:59.524539+0000","last_fullsized":"2023-11-28T11:18:59.524539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:24.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498189+0000","last_change":"2023-11-28T11:18:59.523567+0000","last_active":"2023-11-28T11:19:00.498189+0000","last_peered":"2023-11-28T11:19:00.498189+0000","last_clean":"2023-11-28T11:19:00.498189+0000","last_became_active":"2023-11-28T11:18:59.520356+0000","last_became_peered":"2023-11-28T11:18:59.520356+0000","last_unstale":"2023-11-28T11:19:00.498189+0000","last_undegraded":"2023-11-28T11:19:00.498189+0000","last_fullsized":"2023-11-28T11:19:00.498189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:07.263691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497529+0000","last_change":"2023-11-28T11:18:58.985409+0000","last_active":"2023-11-28T11:19:00.497529+0000","last_peered":"2023-11-28T11:19:00.497529+0000","last_clean":"2023-11-28T11:19:00.497529+0000","last_became_active":"2023-11-28T11:18:58.983943+0000","last_became_peered":"2023-11-28T11:18:58.983943+0000","last_unstale":"2023-11-28T11:19:00.497529+0000","last_undegraded":"2023-11-28T11:19:00.497529+0000","last_fullsized":"2023-11-28T11:19:00.497529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:31.307853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522173+0000","last_change":"2023-11-28T11:18:59.521266+0000","last_active":"2023-11-28T11:18:59.522173+0000","last_peered":"2023-11-28T11:18:59.522173+0000","last_clean":"2023-11-28T11:18:59.522173+0000","last_became_active":"2023-11-28T11:18:59.519421+0000","last_became_peered":"2023-11-28T11:18:59.519421+0000","last_unstale":"2023-11-28T11:18:59.522173+0000","last_undegraded":"2023-11-28T11:18:59.522173+0000","last_fullsized":"2023-11-28T11:18:59.522173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498237+0000","last_change":"2023-11-28T11:18:58.987642+0000","last_active":"2023-11-28T11:19:00.498237+0000","last_peered":"2023-11-28T11:19:00.498237+0000","last_clean":"2023-11-28T11:19:00.498237+0000","last_became_active":"2023-11-28T11:18:58.987103+0000","last_became_peered":"2023-11-28T11:18:58.987103+0000","last_unstale":"2023-11-28T11:19:00.498237+0000","last_undegraded":"2023-11-28T11:19:00.498237+0000","last_fullsized":"2023-11-28T11:19:00.498237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.385763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.516606+0000","last_change":"2023-11-28T11:18:59.514522+0000","last_active":"2023-11-28T11:18:59.516606+0000","last_peered":"2023-11-28T11:18:59.516606+0000","last_clean":"2023-11-28T11:18:59.516606+0000","last_became_active":"2023-11-28T11:18:59.514262+0000","last_became_peered":"2023-11-28T11:18:59.514262+0000","last_unstale":"2023-11-28T11:18:59.516606+0000","last_undegraded":"2023-11-28T11:18:59.516606+0000","last_fullsized":"2023-11-28T11:18:59.516606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498696+0000","last_change":"2023-11-28T11:18:58.988031+0000","last_active":"2023-11-28T11:19:00.498696+0000","last_peered":"2023-11-28T11:19:00.498696+0000","last_clean":"2023-11-28T11:19:00.498696+0000","last_became_active":"2023-11-28T11:18:58.987733+0000","last_became_peered":"2023-11-28T11:18:58.987733+0000","last_unstale":"2023-11-28T11:19:00.498696+0000","last_undegraded":"2023-11-28T11:19:00.498696+0000","last_fullsized":"2023-11-28T11:19:00.498696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:45.176553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498682+0000","last_change":"2023-11-28T11:18:59.516963+0000","last_active":"2023-11-28T11:19:00.498682+0000","last_peered":"2023-11-28T11:19:00.498682+0000","last_clean":"2023-11-28T11:19:00.498682+0000","last_became_active":"2023-11-28T11:18:59.516573+0000","last_became_peered":"2023-11-28T11:18:59.516573+0000","last_unstale":"2023-11-28T11:19:00.498682+0000","last_undegraded":"2023-11-28T11:19:00.498682+0000","last_fullsized":"2023-11-28T11:19:00.498682+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:54.102035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524318+0000","last_change":"2023-11-28T11:18:58.994960+0000","last_active":"2023-11-28T11:18:59.524318+0000","last_peered":"2023-11-28T11:18:59.524318+0000","last_clean":"2023-11-28T11:18:59.524318+0000","last_became_active":"2023-11-28T11:18:58.994684+0000","last_became_peered":"2023-11-28T11:18:58.994684+0000","last_unstale":"2023-11-28T11:18:59.524318+0000","last_undegraded":"2023-11-28T11:18:59.524318+0000","last_fullsized":"2023-11-28T11:18:59.524318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:00.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498005+0000","last_change":"2023-11-28T11:18:59.516604+0000","last_active":"2023-11-28T11:19:00.498005+0000","last_peered":"2023-11-28T11:19:00.498005+0000","last_clean":"2023-11-28T11:19:00.498005+0000","last_became_active":"2023-11-28T11:18:59.516286+0000","last_became_peered":"2023-11-28T11:18:59.516286+0000","last_unstale":"2023-11-28T11:19:00.498005+0000","last_undegraded":"2023-11-28T11:19:00.498005+0000","last_fullsized":"2023-11-28T11:19:00.498005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:20:49.583622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497253+0000","last_change":"2023-11-28T11:18:58.995233+0000","last_active":"2023-11-28T11:19:00.497253+0000","last_peered":"2023-11-28T11:19:00.497253+0000","last_clean":"2023-11-28T11:19:00.497253+0000","last_became_active":"2023-11-28T11:18:58.994941+0000","last_became_peered":"2023-11-28T11:18:58.994941+0000","last_unstale":"2023-11-28T11:19:00.497253+0000","last_undegraded":"2023-11-28T11:19:00.497253+0000","last_fullsized":"2023-11-28T11:19:00.497253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:29.352602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.513161+0000","last_change":"2023-11-28T11:18:59.504304+0000","last_active":"2023-11-28T11:18:59.513161+0000","last_peered":"2023-11-28T11:18:59.513161+0000","last_clean":"2023-11-28T11:18:59.513161+0000","last_became_active":"2023-11-28T11:18:59.502864+0000","last_became_peered":"2023-11-28T11:18:59.502864+0000","last_unstale":"2023-11-28T11:18:59.513161+0000","last_undegraded":"2023-11-28T11:18:59.513161+0000","last_fullsized":"2023-11-28T11:18:59.513161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:29.741286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497746+0000","last_change":"2023-11-28T11:18:58.989845+0000","last_active":"2023-11-28T11:19:00.497746+0000","last_peered":"2023-11-28T11:19:00.497746+0000","last_clean":"2023-11-28T11:19:00.497746+0000","last_became_active":"2023-11-28T11:18:58.989473+0000","last_became_peered":"2023-11-28T11:18:58.989473+0000","last_unstale":"2023-11-28T11:19:00.497746+0000","last_undegraded":"2023-11-28T11:19:00.497746+0000","last_fullsized":"2023-11-28T11:19:00.497746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:22.877375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.526097+0000","last_change":"2023-11-28T11:18:59.525388+0000","last_active":"2023-11-28T11:18:59.526097+0000","last_peered":"2023-11-28T11:18:59.526097+0000","last_clean":"2023-11-28T11:18:59.526097+0000","last_became_active":"2023-11-28T11:18:59.525136+0000","last_became_peered":"2023-11-28T11:18:59.525136+0000","last_unstale":"2023-11-28T11:18:59.526097+0000","last_undegraded":"2023-11-28T11:18:59.526097+0000","last_fullsized":"2023-11-28T11:18:59.526097+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:01.731030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.495889+0000","last_change":"2023-11-28T11:18:58.989231+0000","last_active":"2023-11-28T11:18:59.495889+0000","last_peered":"2023-11-28T11:18:59.495889+0000","last_clean":"2023-11-28T11:18:59.495889+0000","last_became_active":"2023-11-28T11:18:58.988914+0000","last_became_peered":"2023-11-28T11:18:58.988914+0000","last_unstale":"2023-11-28T11:18:59.495889+0000","last_undegraded":"2023-11-28T11:18:59.495889+0000","last_fullsized":"2023-11-28T11:18:59.495889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:03.333903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.526438+0000","last_change":"2023-11-28T11:18:59.521471+0000","last_active":"2023-11-28T11:18:59.526438+0000","last_peered":"2023-11-28T11:18:59.526438+0000","last_clean":"2023-11-28T11:18:59.526438+0000","last_became_active":"2023-11-28T11:18:59.521210+0000","last_became_peered":"2023-11-28T11:18:59.521210+0000","last_unstale":"2023-11-28T11:18:59.526438+0000","last_undegraded":"2023-11-28T11:18:59.526438+0000","last_fullsized":"2023-11-28T11:18:59.526438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:29.741286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499049+0000","last_change":"2023-11-28T11:18:58.997601+0000","last_active":"2023-11-28T11:19:00.499049+0000","last_peered":"2023-11-28T11:19:00.499049+0000","last_clean":"2023-11-28T11:19:00.499049+0000","last_became_active":"2023-11-28T11:18:58.997116+0000","last_became_peered":"2023-11-28T11:18:58.997116+0000","last_unstale":"2023-11-28T11:19:00.499049+0000","last_undegraded":"2023-11-28T11:19:00.499049+0000","last_fullsized":"2023-11-28T11:19:00.499049+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:06.439357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515402+0000","last_change":"2023-11-28T11:18:59.508337+0000","last_active":"2023-11-28T11:18:59.515402+0000","last_peered":"2023-11-28T11:18:59.515402+0000","last_clean":"2023-11-28T11:18:59.515402+0000","last_became_active":"2023-11-28T11:18:59.508143+0000","last_became_peered":"2023-11-28T11:18:59.508143+0000","last_unstale":"2023-11-28T11:18:59.515402+0000","last_undegraded":"2023-11-28T11:18:59.515402+0000","last_fullsized":"2023-11-28T11:18:59.515402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499237+0000","last_change":"2023-11-28T11:18:58.992564+0000","last_active":"2023-11-28T11:19:00.499237+0000","last_peered":"2023-11-28T11:19:00.499237+0000","last_clean":"2023-11-28T11:19:00.499237+0000","last_became_active":"2023-11-28T11:18:58.992295+0000","last_became_peered":"2023-11-28T11:18:58.992295+0000","last_unstale":"2023-11-28T11:19:00.499237+0000","last_undegraded":"2023-11-28T11:19:00.499237+0000","last_fullsized":"2023-11-28T11:19:00.499237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:41.179640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498484+0000","last_change":"2023-11-28T11:18:59.519390+0000","last_active":"2023-11-28T11:19:00.498484+0000","last_peered":"2023-11-28T11:19:00.498484+0000","last_clean":"2023-11-28T11:19:00.498484+0000","last_became_active":"2023-11-28T11:18:59.519109+0000","last_became_peered":"2023-11-28T11:18:59.519109+0000","last_unstale":"2023-11-28T11:19:00.498484+0000","last_undegraded":"2023-11-28T11:19:00.498484+0000","last_fullsized":"2023-11-28T11:19:00.498484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:18.186373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532999+0000","last_change":"2023-11-28T11:18:58.990113+0000","last_active":"2023-11-28T11:18:59.532999+0000","last_peered":"2023-11-28T11:18:59.532999+0000","last_clean":"2023-11-28T11:18:59.532999+0000","last_became_active":"2023-11-28T11:18:58.989849+0000","last_became_peered":"2023-11-28T11:18:58.989849+0000","last_unstale":"2023-11-28T11:18:59.532999+0000","last_undegraded":"2023-11-28T11:18:59.532999+0000","last_fullsized":"2023-11-28T11:18:59.532999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:34.047643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532252+0000","last_change":"2023-11-28T11:18:59.529632+0000","last_active":"2023-11-28T11:18:59.532252+0000","last_peered":"2023-11-28T11:18:59.532252+0000","last_clean":"2023-11-28T11:18:59.532252+0000","last_became_active":"2023-11-28T11:18:59.529360+0000","last_became_peered":"2023-11-28T11:18:59.529360+0000","last_unstale":"2023-11-28T11:18:59.532252+0000","last_undegraded":"2023-11-28T11:18:59.532252+0000","last_fullsized":"2023-11-28T11:18:59.532252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.368483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517825+0000","last_change":"2023-11-28T11:18:58.987543+0000","last_active":"2023-11-28T11:18:59.517825+0000","last_peered":"2023-11-28T11:18:59.517825+0000","last_clean":"2023-11-28T11:18:59.517825+0000","last_became_active":"2023-11-28T11:18:58.987071+0000","last_became_peered":"2023-11-28T11:18:58.987071+0000","last_unstale":"2023-11-28T11:18:59.517825+0000","last_undegraded":"2023-11-28T11:18:59.517825+0000","last_fullsized":"2023-11-28T11:18:59.517825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:37.461831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517583+0000","last_change":"2023-11-28T11:18:59.516217+0000","last_active":"2023-11-28T11:18:59.517583+0000","last_peered":"2023-11-28T11:18:59.517583+0000","last_clean":"2023-11-28T11:18:59.517583+0000","last_became_active":"2023-11-28T11:18:59.516020+0000","last_became_peered":"2023-11-28T11:18:59.516020+0000","last_unstale":"2023-11-28T11:18:59.517583+0000","last_undegraded":"2023-11-28T11:18:59.517583+0000","last_fullsized":"2023-11-28T11:18:59.517583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497102+0000","last_change":"2023-11-28T11:18:58.999366+0000","last_active":"2023-11-28T11:19:00.497102+0000","last_peered":"2023-11-28T11:19:00.497102+0000","last_clean":"2023-11-28T11:19:00.497102+0000","last_became_active":"2023-11-28T11:18:58.999074+0000","last_became_peered":"2023-11-28T11:18:58.999074+0000","last_unstale":"2023-11-28T11:19:00.497102+0000","last_undegraded":"2023-11-28T11:19:00.497102+0000","last_fullsized":"2023-11-28T11:19:00.497102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:52.844122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524386+0000","last_change":"2023-11-28T11:18:59.518072+0000","last_active":"2023-11-28T11:18:59.524386+0000","last_peered":"2023-11-28T11:18:59.524386+0000","last_clean":"2023-11-28T11:18:59.524386+0000","last_became_active":"2023-11-28T11:18:59.517757+0000","last_became_peered":"2023-11-28T11:18:59.517757+0000","last_unstale":"2023-11-28T11:18:59.524386+0000","last_undegraded":"2023-11-28T11:18:59.524386+0000","last_fullsized":"2023-11-28T11:18:59.524386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:44.392064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747387+0000","last_change":"2023-11-28T11:18:58.988319+0000","last_active":"2023-11-28T11:19:00.747387+0000","last_peered":"2023-11-28T11:19:00.747387+0000","last_clean":"2023-11-28T11:19:00.747387+0000","last_became_active":"2023-11-28T11:18:58.986732+0000","last_became_peered":"2023-11-28T11:18:58.986732+0000","last_unstale":"2023-11-28T11:19:00.747387+0000","last_undegraded":"2023-11-28T11:19:00.747387+0000","last_fullsized":"2023-11-28T11:19:00.747387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:26.328175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.514444+0000","last_change":"2023-11-28T11:18:59.504921+0000","last_active":"2023-11-28T11:18:59.514444+0000","last_peered":"2023-11-28T11:18:59.514444+0000","last_clean":"2023-11-28T11:18:59.514444+0000","last_became_active":"2023-11-28T11:18:59.504639+0000","last_became_peered":"2023-11-28T11:18:59.504639+0000","last_unstale":"2023-11-28T11:18:59.514444+0000","last_undegraded":"2023-11-28T11:18:59.514444+0000","last_fullsized":"2023-11-28T11:18:59.514444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:08:00.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.529710+0000","last_change":"2023-11-28T11:18:58.992215+0000","last_active":"2023-11-28T11:18:59.529710+0000","last_peered":"2023-11-28T11:18:59.529710+0000","last_clean":"2023-11-28T11:18:59.529710+0000","last_became_active":"2023-11-28T11:18:58.991938+0000","last_became_peered":"2023-11-28T11:18:58.991938+0000","last_unstale":"2023-11-28T11:18:59.529710+0000","last_undegraded":"2023-11-28T11:18:59.529710+0000","last_fullsized":"2023-11-28T11:18:59.529710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:29.352602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519060+0000","last_change":"2023-11-28T11:18:59.517000+0000","last_active":"2023-11-28T11:18:59.519060+0000","last_peered":"2023-11-28T11:18:59.519060+0000","last_clean":"2023-11-28T11:18:59.519060+0000","last_became_active":"2023-11-28T11:18:59.515317+0000","last_became_peered":"2023-11-28T11:18:59.515317+0000","last_unstale":"2023-11-28T11:18:59.519060+0000","last_undegraded":"2023-11-28T11:18:59.519060+0000","last_fullsized":"2023-11-28T11:18:59.519060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499484+0000","last_change":"2023-11-28T11:18:58.994494+0000","last_active":"2023-11-28T11:19:00.499484+0000","last_peered":"2023-11-28T11:19:00.499484+0000","last_clean":"2023-11-28T11:19:00.499484+0000","last_became_active":"2023-11-28T11:18:58.993842+0000","last_became_peered":"2023-11-28T11:18:58.993842+0000","last_unstale":"2023-11-28T11:19:00.499484+0000","last_undegraded":"2023-11-28T11:19:00.499484+0000","last_fullsized":"2023-11-28T11:19:00.499484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:44.182419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499129+0000","last_change":"2023-11-28T11:18:59.527401+0000","last_active":"2023-11-28T11:19:00.499129+0000","last_peered":"2023-11-28T11:19:00.499129+0000","last_clean":"2023-11-28T11:19:00.499129+0000","last_became_active":"2023-11-28T11:18:59.524390+0000","last_became_peered":"2023-11-28T11:18:59.524390+0000","last_unstale":"2023-11-28T11:19:00.499129+0000","last_undegraded":"2023-11-28T11:19:00.499129+0000","last_fullsized":"2023-11-28T11:19:00.499129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:55.258780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498415+0000","last_change":"2023-11-28T11:18:58.991079+0000","last_active":"2023-11-28T11:19:00.498415+0000","last_peered":"2023-11-28T11:19:00.498415+0000","last_clean":"2023-11-28T11:19:00.498415+0000","last_became_active":"2023-11-28T11:18:58.990803+0000","last_became_peered":"2023-11-28T11:18:58.990803+0000","last_unstale":"2023-11-28T11:19:00.498415+0000","last_undegraded":"2023-11-28T11:19:00.498415+0000","last_fullsized":"2023-11-28T11:19:00.498415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:28.734985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498301+0000","last_change":"2023-11-28T11:18:59.506613+0000","last_active":"2023-11-28T11:19:00.498301+0000","last_peered":"2023-11-28T11:19:00.498301+0000","last_clean":"2023-11-28T11:19:00.498301+0000","last_became_active":"2023-11-28T11:18:59.506258+0000","last_became_peered":"2023-11-28T11:18:59.506258+0000","last_unstale":"2023-11-28T11:19:00.498301+0000","last_undegraded":"2023-11-28T11:19:00.498301+0000","last_fullsized":"2023-11-28T11:19:00.498301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:38.099722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.529893+0000","last_change":"2023-11-28T11:18:58.990358+0000","last_active":"2023-11-28T11:18:59.529893+0000","last_peered":"2023-11-28T11:18:59.529893+0000","last_clean":"2023-11-28T11:18:59.529893+0000","last_became_active":"2023-11-28T11:18:58.990090+0000","last_became_peered":"2023-11-28T11:18:58.990090+0000","last_unstale":"2023-11-28T11:18:59.529893+0000","last_undegraded":"2023-11-28T11:18:59.529893+0000","last_fullsized":"2023-11-28T11:18:59.529893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:20.314008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.521497+0000","last_change":"2023-11-28T11:18:59.519405+0000","last_active":"2023-11-28T11:18:59.521497+0000","last_peered":"2023-11-28T11:18:59.521497+0000","last_clean":"2023-11-28T11:18:59.521497+0000","last_became_active":"2023-11-28T11:18:59.519143+0000","last_became_peered":"2023-11-28T11:18:59.519143+0000","last_unstale":"2023-11-28T11:18:59.521497+0000","last_undegraded":"2023-11-28T11:18:59.521497+0000","last_fullsized":"2023-11-28T11:18:59.521497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:53.672693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497124+0000","last_change":"2023-11-28T11:18:58.984350+0000","last_active":"2023-11-28T11:19:00.497124+0000","last_peered":"2023-11-28T11:19:00.497124+0000","last_clean":"2023-11-28T11:19:00.497124+0000","last_became_active":"2023-11-28T11:18:58.983962+0000","last_became_peered":"2023-11-28T11:18:58.983962+0000","last_unstale":"2023-11-28T11:19:00.497124+0000","last_undegraded":"2023-11-28T11:19:00.497124+0000","last_fullsized":"2023-11-28T11:19:00.497124+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:15.858163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.523362+0000","last_change":"2023-11-28T11:18:59.515035+0000","last_active":"2023-11-28T11:18:59.523362+0000","last_peered":"2023-11-28T11:18:59.523362+0000","last_clean":"2023-11-28T11:18:59.523362+0000","last_became_active":"2023-11-28T11:18:59.514703+0000","last_became_peered":"2023-11-28T11:18:59.514703+0000","last_unstale":"2023-11-28T11:18:59.523362+0000","last_undegraded":"2023-11-28T11:18:59.523362+0000","last_fullsized":"2023-11-28T11:18:59.523362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519675+0000","last_change":"2023-11-28T11:18:58.981751+0000","last_active":"2023-11-28T11:18:59.519675+0000","last_peered":"2023-11-28T11:18:59.519675+0000","last_clean":"2023-11-28T11:18:59.519675+0000","last_became_active":"2023-11-28T11:18:58.981450+0000","last_became_peered":"2023-11-28T11:18:58.981450+0000","last_unstale":"2023-11-28T11:18:59.519675+0000","last_undegraded":"2023-11-28T11:18:59.519675+0000","last_fullsized":"2023-11-28T11:18:59.519675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:21.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515530+0000","last_change":"2023-11-28T11:18:59.507434+0000","last_active":"2023-11-28T11:18:59.515530+0000","last_peered":"2023-11-28T11:18:59.515530+0000","last_clean":"2023-11-28T11:18:59.515530+0000","last_became_active":"2023-11-28T11:18:59.507089+0000","last_became_peered":"2023-11-28T11:18:59.507089+0000","last_unstale":"2023-11-28T11:18:59.515530+0000","last_undegraded":"2023-11-28T11:18:59.515530+0000","last_fullsized":"2023-11-28T11:18:59.515530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.509773+0000","last_change":"2023-11-28T11:18:58.977878+0000","last_active":"2023-11-28T11:18:59.509773+0000","last_peered":"2023-11-28T11:18:59.509773+0000","last_clean":"2023-11-28T11:18:59.509773+0000","last_became_active":"2023-11-28T11:18:58.977593+0000","last_became_peered":"2023-11-28T11:18:58.977593+0000","last_unstale":"2023-11-28T11:18:59.509773+0000","last_undegraded":"2023-11-28T11:18:59.509773+0000","last_fullsized":"2023-11-28T11:18:59.509773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:36.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498692+0000","last_change":"2023-11-28T11:18:59.528812+0000","last_active":"2023-11-28T11:19:00.498692+0000","last_peered":"2023-11-28T11:19:00.498692+0000","last_clean":"2023-11-28T11:19:00.498692+0000","last_became_active":"2023-11-28T11:18:59.528546+0000","last_became_peered":"2023-11-28T11:18:59.528546+0000","last_unstale":"2023-11-28T11:19:00.498692+0000","last_undegraded":"2023-11-28T11:19:00.498692+0000","last_fullsized":"2023-11-28T11:19:00.498692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:36.838348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.499374+0000","last_change":"2023-11-28T11:18:58.994878+0000","last_active":"2023-11-28T11:18:59.499374+0000","last_peered":"2023-11-28T11:18:59.499374+0000","last_clean":"2023-11-28T11:18:59.499374+0000","last_became_active":"2023-11-28T11:18:58.994618+0000","last_became_peered":"2023-11-28T11:18:58.994618+0000","last_unstale":"2023-11-28T11:18:59.499374+0000","last_undegraded":"2023-11-28T11:18:59.499374+0000","last_fullsized":"2023-11-28T11:18:59.499374+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:03.333903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515831+0000","last_change":"2023-11-28T11:18:59.508311+0000","last_active":"2023-11-28T11:18:59.515831+0000","last_peered":"2023-11-28T11:18:59.515831+0000","last_clean":"2023-11-28T11:18:59.515831+0000","last_became_active":"2023-11-28T11:18:59.508042+0000","last_became_peered":"2023-11-28T11:18:59.508042+0000","last_unstale":"2023-11-28T11:18:59.515831+0000","last_undegraded":"2023-11-28T11:18:59.515831+0000","last_fullsized":"2023-11-28T11:18:59.515831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.509884+0000","last_change":"2023-11-28T11:18:58.981912+0000","last_active":"2023-11-28T11:18:59.509884+0000","last_peered":"2023-11-28T11:18:59.509884+0000","last_clean":"2023-11-28T11:18:59.509884+0000","last_became_active":"2023-11-28T11:18:58.981302+0000","last_became_peered":"2023-11-28T11:18:58.981302+0000","last_unstale":"2023-11-28T11:18:59.509884+0000","last_undegraded":"2023-11-28T11:18:59.509884+0000","last_fullsized":"2023-11-28T11:18:59.509884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:01:20.118399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515648+0000","last_change":"2023-11-28T11:18:59.509879+0000","last_active":"2023-11-28T11:18:59.515648+0000","last_peered":"2023-11-28T11:18:59.515648+0000","last_clean":"2023-11-28T11:18:59.515648+0000","last_became_active":"2023-11-28T11:18:59.507927+0000","last_became_peered":"2023-11-28T11:18:59.507927+0000","last_unstale":"2023-11-28T11:18:59.515648+0000","last_undegraded":"2023-11-28T11:18:59.515648+0000","last_fullsized":"2023-11-28T11:18:59.515648+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:07.099752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.501978+0000","last_change":"2023-11-28T11:18:58.988242+0000","last_active":"2023-11-28T11:18:59.501978+0000","last_peered":"2023-11-28T11:18:59.501978+0000","last_clean":"2023-11-28T11:18:59.501978+0000","last_became_active":"2023-11-28T11:18:58.987966+0000","last_became_peered":"2023-11-28T11:18:58.987966+0000","last_unstale":"2023-11-28T11:18:59.501978+0000","last_undegraded":"2023-11-28T11:18:59.501978+0000","last_fullsized":"2023-11-28T11:18:59.501978+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:52.666393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.504047+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:18:59.504047+0000","last_peered":"2023-11-28T11:18:59.504047+0000","last_clean":"2023-11-28T11:18:59.504047+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:18:59.504047+0000","last_undegraded":"2023-11-28T11:18:59.504047+0000","last_fullsized":"2023-11-28T11:18:59.504047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.512100+0000","last_change":"2023-11-28T11:18:59.503838+0000","last_active":"2023-11-28T11:18:59.512100+0000","last_peered":"2023-11-28T11:18:59.512100+0000","last_clean":"2023-11-28T11:18:59.512100+0000","last_became_active":"2023-11-28T11:18:59.503552+0000","last_became_peered":"2023-11-28T11:18:59.503552+0000","last_unstale":"2023-11-28T11:18:59.512100+0000","last_undegraded":"2023-11-28T11:18:59.512100+0000","last_fullsized":"2023-11-28T11:18:59.512100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.507576+0000","last_change":"2023-11-28T11:18:58.984310+0000","last_active":"2023-11-28T11:18:59.507576+0000","last_peered":"2023-11-28T11:18:59.507576+0000","last_clean":"2023-11-28T11:18:59.507576+0000","last_became_active":"2023-11-28T11:18:58.984049+0000","last_became_peered":"2023-11-28T11:18:58.984049+0000","last_unstale":"2023-11-28T11:18:59.507576+0000","last_undegraded":"2023-11-28T11:18:59.507576+0000","last_fullsized":"2023-11-28T11:18:59.507576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:37.461831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.510427+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:18:59.510427+0000","last_peered":"2023-11-28T11:18:59.510427+0000","last_clean":"2023-11-28T11:18:59.510427+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:18:59.510427+0000","last_undegraded":"2023-11-28T11:18:59.510427+0000","last_fullsized":"2023-11-28T11:18:59.510427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525018+0000","last_change":"2023-11-28T11:18:59.516005+0000","last_active":"2023-11-28T11:18:59.525018+0000","last_peered":"2023-11-28T11:18:59.525018+0000","last_clean":"2023-11-28T11:18:59.525018+0000","last_became_active":"2023-11-28T11:18:59.515002+0000","last_became_peered":"2023-11-28T11:18:59.515002+0000","last_unstale":"2023-11-28T11:18:59.525018+0000","last_undegraded":"2023-11-28T11:18:59.525018+0000","last_fullsized":"2023-11-28T11:18:59.525018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:50.802855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.510660+0000","last_change":"2023-11-28T11:18:58.987720+0000","last_active":"2023-11-28T11:18:59.510660+0000","last_peered":"2023-11-28T11:18:59.510660+0000","last_clean":"2023-11-28T11:18:59.510660+0000","last_became_active":"2023-11-28T11:18:58.987490+0000","last_became_peered":"2023-11-28T11:18:58.987490+0000","last_unstale":"2023-11-28T11:18:59.510660+0000","last_undegraded":"2023-11-28T11:18:59.510660+0000","last_fullsized":"2023-11-28T11:18:59.510660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:52.951851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.977131+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:18:58.977131+0000","last_peered":"2023-11-28T11:18:58.977131+0000","last_clean":"2023-11-28T11:18:58.977131+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:18:58.977131+0000","last_undegraded":"2023-11-28T11:18:58.977131+0000","last_fullsized":"2023-11-28T11:18:58.977131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499438+0000","last_change":"2023-11-28T11:18:59.525045+0000","last_active":"2023-11-28T11:19:00.499438+0000","last_peered":"2023-11-28T11:19:00.499438+0000","last_clean":"2023-11-28T11:19:00.499438+0000","last_became_active":"2023-11-28T11:18:59.524541+0000","last_became_peered":"2023-11-28T11:18:59.524541+0000","last_unstale":"2023-11-28T11:19:00.499438+0000","last_undegraded":"2023-11-28T11:19:00.499438+0000","last_fullsized":"2023-11-28T11:19:00.499438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:57.492401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.498893+0000","last_change":"2023-11-28T11:18:58.996898+0000","last_active":"2023-11-28T11:18:59.498893+0000","last_peered":"2023-11-28T11:18:59.498893+0000","last_clean":"2023-11-28T11:18:59.498893+0000","last_became_active":"2023-11-28T11:18:58.996609+0000","last_became_peered":"2023-11-28T11:18:58.996609+0000","last_unstale":"2023-11-28T11:18:59.498893+0000","last_undegraded":"2023-11-28T11:18:59.498893+0000","last_fullsized":"2023-11-28T11:18:59.498893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:27.115435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.513787+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:18:59.513787+0000","last_peered":"2023-11-28T11:18:59.513787+0000","last_clean":"2023-11-28T11:18:59.513787+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:18:59.513787+0000","last_undegraded":"2023-11-28T11:18:59.513787+0000","last_fullsized":"2023-11-28T11:18:59.513787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.984716+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:18:58.984716+0000","last_peered":"2023-11-28T11:18:58.984716+0000","last_clean":"2023-11-28T11:18:58.984716+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:18:58.984716+0000","last_undegraded":"2023-11-28T11:18:58.984716+0000","last_fullsized":"2023-11-28T11:18:58.984716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497241+0000","last_change":"2023-11-28T11:18:59.529816+0000","last_active":"2023-11-28T11:19:00.497241+0000","last_peered":"2023-11-28T11:19:00.497241+0000","last_clean":"2023-11-28T11:19:00.497241+0000","last_became_active":"2023-11-28T11:18:59.529201+0000","last_became_peered":"2023-11-28T11:18:59.529201+0000","last_unstale":"2023-11-28T11:19:00.497241+0000","last_undegraded":"2023-11-28T11:19:00.497241+0000","last_fullsized":"2023-11-28T11:19:00.497241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:07.099752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499495+0000","last_change":"2023-11-28T11:18:58.998296+0000","last_active":"2023-11-28T11:19:00.499495+0000","last_peered":"2023-11-28T11:19:00.499495+0000","last_clean":"2023-11-28T11:19:00.499495+0000","last_became_active":"2023-11-28T11:18:58.997905+0000","last_became_peered":"2023-11-28T11:18:58.997905+0000","last_unstale":"2023-11-28T11:19:00.499495+0000","last_undegraded":"2023-11-28T11:19:00.499495+0000","last_fullsized":"2023-11-28T11:19:00.499495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:29.967341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.985447+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:18:58.985447+0000","last_peered":"2023-11-28T11:18:58.985447+0000","last_clean":"2023-11-28T11:18:58.985447+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:18:58.985447+0000","last_undegraded":"2023-11-28T11:18:58.985447+0000","last_fullsized":"2023-11-28T11:18:58.985447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.496962+0000","last_change":"2023-11-28T11:18:59.528417+0000","last_active":"2023-11-28T11:19:00.496962+0000","last_peered":"2023-11-28T11:19:00.496962+0000","last_clean":"2023-11-28T11:19:00.496962+0000","last_became_active":"2023-11-28T11:18:59.528158+0000","last_became_peered":"2023-11-28T11:18:59.528158+0000","last_unstale":"2023-11-28T11:19:00.496962+0000","last_undegraded":"2023-11-28T11:19:00.496962+0000","last_fullsized":"2023-11-28T11:19:00.496962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:40.829763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747504+0000","last_change":"2023-11-28T11:18:58.985101+0000","last_active":"2023-11-28T11:19:00.747504+0000","last_peered":"2023-11-28T11:19:00.747504+0000","last_clean":"2023-11-28T11:19:00.747504+0000","last_became_active":"2023-11-28T11:18:58.984641+0000","last_became_peered":"2023-11-28T11:18:58.984641+0000","last_unstale":"2023-11-28T11:19:00.747504+0000","last_undegraded":"2023-11-28T11:19:00.747504+0000","last_fullsized":"2023-11-28T11:19:00.747504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:31.036152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.993331+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:18:58.993331+0000","last_peered":"2023-11-28T11:18:58.993331+0000","last_clean":"2023-11-28T11:18:58.993331+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:18:58.993331+0000","last_undegraded":"2023-11-28T11:18:58.993331+0000","last_fullsized":"2023-11-28T11:18:58.993331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498131+0000","last_change":"2023-11-28T11:18:59.503325+0000","last_active":"2023-11-28T11:19:00.498131+0000","last_peered":"2023-11-28T11:19:00.498131+0000","last_clean":"2023-11-28T11:19:00.498131+0000","last_became_active":"2023-11-28T11:18:59.502962+0000","last_became_peered":"2023-11-28T11:18:59.502962+0000","last_unstale":"2023-11-28T11:19:00.498131+0000","last_undegraded":"2023-11-28T11:19:00.498131+0000","last_fullsized":"2023-11-28T11:19:00.498131+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:32.721152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498406+0000","last_change":"2023-11-28T11:18:58.985530+0000","last_active":"2023-11-28T11:19:00.498406+0000","last_peered":"2023-11-28T11:19:00.498406+0000","last_clean":"2023-11-28T11:19:00.498406+0000","last_became_active":"2023-11-28T11:18:58.985204+0000","last_became_peered":"2023-11-28T11:18:58.985204+0000","last_unstale":"2023-11-28T11:19:00.498406+0000","last_undegraded":"2023-11-28T11:19:00.498406+0000","last_fullsized":"2023-11-28T11:19:00.498406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:49.796554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.503281+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:18:59.503281+0000","last_peered":"2023-11-28T11:18:59.503281+0000","last_clean":"2023-11-28T11:18:59.503281+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:18:59.503281+0000","last_undegraded":"2023-11-28T11:18:59.503281+0000","last_fullsized":"2023-11-28T11:18:59.503281+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522467+0000","last_change":"2023-11-28T11:18:59.514225+0000","last_active":"2023-11-28T11:18:59.522467+0000","last_peered":"2023-11-28T11:18:59.522467+0000","last_clean":"2023-11-28T11:18:59.522467+0000","last_became_active":"2023-11-28T11:18:59.513872+0000","last_became_peered":"2023-11-28T11:18:59.513872+0000","last_unstale":"2023-11-28T11:18:59.522467+0000","last_undegraded":"2023-11-28T11:18:59.522467+0000","last_fullsized":"2023-11-28T11:18:59.522467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.368483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.494843+0000","last_change":"2023-11-28T11:18:58.990718+0000","last_active":"2023-11-28T11:18:59.494843+0000","last_peered":"2023-11-28T11:18:59.494843+0000","last_clean":"2023-11-28T11:18:59.494843+0000","last_became_active":"2023-11-28T11:18:58.990168+0000","last_became_peered":"2023-11-28T11:18:58.990168+0000","last_unstale":"2023-11-28T11:18:59.494843+0000","last_undegraded":"2023-11-28T11:18:59.494843+0000","last_fullsized":"2023-11-28T11:18:59.494843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:27.115435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525692+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:18:59.525692+0000","last_peered":"2023-11-28T11:18:59.525692+0000","last_clean":"2023-11-28T11:18:59.525692+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:18:59.525692+0000","last_undegraded":"2023-11-28T11:18:59.525692+0000","last_fullsized":"2023-11-28T11:18:59.525692+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13660,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358180,"statfs":{"total":96636764160,"available":96622776320,"internally_reserved":0,"allocated":1003520,"data_stored":753902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77900000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71499999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12440,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359400,"statfs":{"total":96636764160,"available":96624025600,"internally_reserved":0,"allocated":409600,"data_stored":169135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.76100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.1669999999999998,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.81599999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.95999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67200000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12564,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359276,"statfs":{"total":96636764160,"available":96623898624,"internally_reserved":0,"allocated":405504,"data_stored":167695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.60299999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13580,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94358260,"statfs":{"total":96636764160,"available":96622858240,"internally_reserved":0,"allocated":987136,"data_stored":751299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.50900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67400000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12428,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359412,"statfs":{"total":96636764160,"available":96624037888,"internally_reserved":0,"allocated":397312,"data_stored":161986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.76600000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.4580000000000002,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.622},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.69799999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12632,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":12224,"kb_avail":94359208,"statfs":{"total":96636764160,"available":96623828992,"internally_reserved":0,"allocated":409600,"data_stored":163498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12517376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.9929999999999999,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.754}]},{"osd":7,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.623}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12700,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359140,"statfs":{"total":96636764160,"available":96623759360,"internally_reserved":0,"allocated":413696,"data_stored":169157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.85099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12572,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359268,"statfs":{"total":96636764160,"available":96623890432,"internally_reserved":0,"allocated":413696,"data_stored":168681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.71699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.63}]},{"osd":2,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.6800000000000002,"5min":2.6800000000000002,"15min":2.6800000000000002},"last":0.64700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:19:03.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:19:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.171+0000 7fb24f9ff700 1 Processor -- start 2023-11-28T11:19:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.171+0000 7fb24f9ff700 1 -- start start 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24f9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24f9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 0x7fb2480cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24f9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2480d1950 con 0x7fb248161340 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24f9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480b5800 con 0x7fb24815c9b0 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24f9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480ce140 con 0x7fb2480cd7c0 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24d79b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24d79b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50554/0 (socket says 172.21.15.53:50554) 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.172+0000 7fb24d79b700 1 -- 172.21.15.53:0/3175559315 learned_addr learned my addr 172.21.15.53:0/3175559315 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24df9c700 1 --2- 172.21.15.53:0/3175559315 >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 0x7fb2480cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24d79b700 1 -- 172.21.15.53:0/3175559315 >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 msgr2=0x7fb2480cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24d79b700 1 --2- 172.21.15.53:0/3175559315 >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 0x7fb2480cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24d79b700 1 -- 172.21.15.53:0/3175559315 >> v1:172.21.15.157:6789/0 conn(0x7fb248161340 legacy=0x7fb2480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24d79b700 1 -- 172.21.15.53:0/3175559315 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480d24f0 con 0x7fb24815c9b0 2023-11-28T11:19:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24df9c700 1 --2- 172.21.15.53:0/3175559315 >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 0x7fb2480cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb24d79b700 1 --2- 172.21.15.53:0/3175559315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb24806c170 tx=0x7fb244009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b1bd488f408d2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb23ffff700 1 -- 172.21.15.53:0/3175559315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244004230 con 0x7fb24815c9b0 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb23ffff700 1 -- 172.21.15.53:0/3175559315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244004390 con 0x7fb24815c9b0 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.173+0000 7fb23ffff700 1 -- 172.21.15.53:0/3175559315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2440036a0 con 0x7fb24815c9b0 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 -- 172.21.15.53:0/3175559315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 msgr2=0x7fb2480b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/3175559315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb24806c170 tx=0x7fb244009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 -- 172.21.15.53:0/3175559315 shutdown_connections 2023-11-28T11:19:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/3175559315 >> v2:172.21.15.157:3300/0 conn(0x7fb2480cd7c0 0x7fb2480cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/3175559315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480b4bf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 -- 172.21.15.53:0/3175559315 >> 172.21.15.53:0/3175559315 conn(0x7fb248072600 msgr2=0x7fb2480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 -- 172.21.15.53:0/3175559315 shutdown_connections 2023-11-28T11:19:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.174+0000 7fb24f9ff700 1 -- 172.21.15.53:0/3175559315 wait complete. 2023-11-28T11:19:04.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.176+0000 7fb24f9ff700 1 Processor -- start 2023-11-28T11:19:04.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.176+0000 7fb24f9ff700 1 -- start start 2023-11-28T11:19:04.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 0x7fb248147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb2480c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2481f2b80 con 0x7fb24815c9b0 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2481546b0 con 0x7fb248161340 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24f9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480c8fb0 con 0x7fb2480cd7c0 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50560/0 (socket says 172.21.15.53:50560) 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 -- 172.21.15.53:0/350808830 learned_addr learned my addr 172.21.15.53:0/350808830 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24d79b700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 0x7fb248147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24df9c700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb2480c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 -- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 msgr2=0x7fb248147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 0x7fb248147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 -- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 msgr2=0x7fb2480c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb2480c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24cf9a700 1 -- 172.21.15.53:0/350808830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb244009550 con 0x7fb24815c9b0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24d79b700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 0x7fb248147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.177+0000 7fb24df9c700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb2480c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb24cf9a700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b4a0 tx=0x7fb23800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238011830 con 0x7fb24815c9b0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb238011990 con 0x7fb24815c9b0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238011d40 con 0x7fb24815c9b0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480d1300 con 0x7fb24815c9b0 2023-11-28T11:19:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.178+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2480b5520 con 0x7fb24815c9b0 2023-11-28T11:19:04.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.179+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2480d15e0 con 0x7fb24815c9b0 2023-11-28T11:19:04.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.180+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb23800f050 con 0x7fb24815c9b0 2023-11-28T11:19:04.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.181+0000 7fb23dffb700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.181+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2380859a0 con 0x7fb24815c9b0 2023-11-28T11:19:04.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.181+0000 7fb24d79b700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.182+0000 7fb24d79b700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 0x7fb22c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb244009130 tx=0x7fb244008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.185+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb238052110 con 0x7fb24815c9b0 2023-11-28T11:19:04.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.383+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2480745c0 con 0x7fb22c0682f0 2023-11-28T11:19:04.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.393+0000 7fb23dffb700 1 -- 172.21.15.53:0/350808830 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310843 (secure 0 0 0) 0x7fb2480745c0 con 0x7fb22c0682f0 2023-11-28T11:19:04.399 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.397+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 msgr2=0x7fb22c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.397+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 0x7fb22c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb244009130 tx=0x7fb244008030 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.397+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 msgr2=0x7fb2480c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.397+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b4a0 tx=0x7fb23800b860 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 shutdown_connections 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2480cd7c0 0x7fb248147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb248161340 0x7fb2480c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 --2- 172.21.15.53:0/350808830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb24815c9b0 0x7fb2480c8410 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 >> 172.21.15.53:0/350808830 conn(0x7fb248072600 msgr2=0x7fb24806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:04.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 shutdown_connections 2023-11-28T11:19:04.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.398+0000 7fb24f9ff700 1 -- 172.21.15.53:0/350808830 wait complete. 2023-11-28T11:19:04.403 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:19:04.418 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-11-28T11:19:02.948220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":102576,"kb_used_data":4336,"kb_used_omap":0,"kb_used_meta":98176,"kb_avail":754872144,"statfs":{"total":773094113280,"available":772989075456,"internally_reserved":0,"allocated":4440064,"data_stored":2505353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":100532224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985850"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517971+0000","last_change":"2023-11-28T11:18:59.513740+0000","last_active":"2023-11-28T11:18:59.517971+0000","last_peered":"2023-11-28T11:18:59.517971+0000","last_clean":"2023-11-28T11:18:59.517971+0000","last_became_active":"2023-11-28T11:18:59.513004+0000","last_became_peered":"2023-11-28T11:18:59.513004+0000","last_unstale":"2023-11-28T11:18:59.517971+0000","last_undegraded":"2023-11-28T11:18:59.517971+0000","last_fullsized":"2023-11-28T11:18:59.517971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:09.605544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519842+0000","last_change":"2023-11-28T11:18:59.518148+0000","last_active":"2023-11-28T11:18:59.519842+0000","last_peered":"2023-11-28T11:18:59.519842+0000","last_clean":"2023-11-28T11:18:59.519842+0000","last_became_active":"2023-11-28T11:18:59.517868+0000","last_became_peered":"2023-11-28T11:18:59.517868+0000","last_unstale":"2023-11-28T11:18:59.519842+0000","last_undegraded":"2023-11-28T11:18:59.519842+0000","last_fullsized":"2023-11-28T11:18:59.519842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532776+0000","last_change":"2023-11-28T11:18:59.530889+0000","last_active":"2023-11-28T11:18:59.532776+0000","last_peered":"2023-11-28T11:18:59.532776+0000","last_clean":"2023-11-28T11:18:59.532776+0000","last_became_active":"2023-11-28T11:18:59.530572+0000","last_became_peered":"2023-11-28T11:18:59.530572+0000","last_unstale":"2023-11-28T11:18:59.532776+0000","last_undegraded":"2023-11-28T11:18:59.532776+0000","last_fullsized":"2023-11-28T11:18:59.532776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.528454+0000","last_change":"2023-11-28T11:18:59.527366+0000","last_active":"2023-11-28T11:18:59.528454+0000","last_peered":"2023-11-28T11:18:59.528454+0000","last_clean":"2023-11-28T11:18:59.528454+0000","last_became_active":"2023-11-28T11:18:59.527133+0000","last_became_peered":"2023-11-28T11:18:59.527133+0000","last_unstale":"2023-11-28T11:18:59.528454+0000","last_undegraded":"2023-11-28T11:18:59.528454+0000","last_fullsized":"2023-11-28T11:18:59.528454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:04.340204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522343+0000","last_change":"2023-11-28T11:18:59.520216+0000","last_active":"2023-11-28T11:18:59.522343+0000","last_peered":"2023-11-28T11:18:59.522343+0000","last_clean":"2023-11-28T11:18:59.522343+0000","last_became_active":"2023-11-28T11:18:59.519741+0000","last_became_peered":"2023-11-28T11:18:59.519741+0000","last_unstale":"2023-11-28T11:18:59.522343+0000","last_undegraded":"2023-11-28T11:18:59.522343+0000","last_fullsized":"2023-11-28T11:18:59.522343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:28.121736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.496887+0000","last_change":"2023-11-28T11:18:59.516703+0000","last_active":"2023-11-28T11:19:00.496887+0000","last_peered":"2023-11-28T11:19:00.496887+0000","last_clean":"2023-11-28T11:19:00.496887+0000","last_became_active":"2023-11-28T11:18:59.516422+0000","last_became_peered":"2023-11-28T11:18:59.516422+0000","last_unstale":"2023-11-28T11:19:00.496887+0000","last_undegraded":"2023-11-28T11:19:00.496887+0000","last_fullsized":"2023-11-28T11:19:00.496887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:54.590713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498567+0000","last_change":"2023-11-28T11:18:59.526139+0000","last_active":"2023-11-28T11:19:00.498567+0000","last_peered":"2023-11-28T11:19:00.498567+0000","last_clean":"2023-11-28T11:19:00.498567+0000","last_became_active":"2023-11-28T11:18:59.525351+0000","last_became_peered":"2023-11-28T11:18:59.525351+0000","last_unstale":"2023-11-28T11:19:00.498567+0000","last_undegraded":"2023-11-28T11:19:00.498567+0000","last_fullsized":"2023-11-28T11:19:00.498567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:29.556331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532936+0000","last_change":"2023-11-28T11:18:59.531153+0000","last_active":"2023-11-28T11:18:59.532936+0000","last_peered":"2023-11-28T11:18:59.532936+0000","last_clean":"2023-11-28T11:18:59.532936+0000","last_became_active":"2023-11-28T11:18:59.528513+0000","last_became_peered":"2023-11-28T11:18:59.528513+0000","last_unstale":"2023-11-28T11:18:59.532936+0000","last_undegraded":"2023-11-28T11:18:59.532936+0000","last_fullsized":"2023-11-28T11:18:59.532936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:53.259503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532719+0000","last_change":"2023-11-28T11:18:59.530510+0000","last_active":"2023-11-28T11:18:59.532719+0000","last_peered":"2023-11-28T11:18:59.532719+0000","last_clean":"2023-11-28T11:18:59.532719+0000","last_became_active":"2023-11-28T11:18:59.530179+0000","last_became_peered":"2023-11-28T11:18:59.530179+0000","last_unstale":"2023-11-28T11:18:59.532719+0000","last_undegraded":"2023-11-28T11:18:59.532719+0000","last_fullsized":"2023-11-28T11:18:59.532719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498958+0000","last_change":"2023-11-28T11:18:59.527077+0000","last_active":"2023-11-28T11:19:00.498958+0000","last_peered":"2023-11-28T11:19:00.498958+0000","last_clean":"2023-11-28T11:19:00.498958+0000","last_became_active":"2023-11-28T11:18:59.526784+0000","last_became_peered":"2023-11-28T11:18:59.526784+0000","last_unstale":"2023-11-28T11:19:00.498958+0000","last_undegraded":"2023-11-28T11:19:00.498958+0000","last_fullsized":"2023-11-28T11:19:00.498958+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:41:04.591178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499193+0000","last_change":"2023-11-28T11:18:59.522768+0000","last_active":"2023-11-28T11:19:00.499193+0000","last_peered":"2023-11-28T11:19:00.499193+0000","last_clean":"2023-11-28T11:19:00.499193+0000","last_became_active":"2023-11-28T11:18:59.522477+0000","last_became_peered":"2023-11-28T11:18:59.522477+0000","last_unstale":"2023-11-28T11:19:00.499193+0000","last_undegraded":"2023-11-28T11:19:00.499193+0000","last_fullsized":"2023-11-28T11:19:00.499193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:28:21.182281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522219+0000","last_change":"2023-11-28T11:18:59.520364+0000","last_active":"2023-11-28T11:18:59.522219+0000","last_peered":"2023-11-28T11:18:59.522219+0000","last_clean":"2023-11-28T11:18:59.522219+0000","last_became_active":"2023-11-28T11:18:59.520093+0000","last_became_peered":"2023-11-28T11:18:59.520093+0000","last_unstale":"2023-11-28T11:18:59.522219+0000","last_undegraded":"2023-11-28T11:18:59.522219+0000","last_fullsized":"2023-11-28T11:18:59.522219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497993+0000","last_change":"2023-11-28T11:18:59.526339+0000","last_active":"2023-11-28T11:19:00.497993+0000","last_peered":"2023-11-28T11:19:00.497993+0000","last_clean":"2023-11-28T11:19:00.497993+0000","last_became_active":"2023-11-28T11:18:59.525957+0000","last_became_peered":"2023-11-28T11:18:59.525957+0000","last_unstale":"2023-11-28T11:19:00.497993+0000","last_undegraded":"2023-11-28T11:19:00.497993+0000","last_fullsized":"2023-11-28T11:19:00.497993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:37.757891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498325+0000","last_change":"2023-11-28T11:18:59.523717+0000","last_active":"2023-11-28T11:19:00.498325+0000","last_peered":"2023-11-28T11:19:00.498325+0000","last_clean":"2023-11-28T11:19:00.498325+0000","last_became_active":"2023-11-28T11:18:59.523454+0000","last_became_peered":"2023-11-28T11:18:59.523454+0000","last_unstale":"2023-11-28T11:19:00.498325+0000","last_undegraded":"2023-11-28T11:19:00.498325+0000","last_fullsized":"2023-11-28T11:19:00.498325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:18.743726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498382+0000","last_change":"2023-11-28T11:18:59.525698+0000","last_active":"2023-11-28T11:19:00.498382+0000","last_peered":"2023-11-28T11:19:00.498382+0000","last_clean":"2023-11-28T11:19:00.498382+0000","last_became_active":"2023-11-28T11:18:59.525004+0000","last_became_peered":"2023-11-28T11:18:59.525004+0000","last_unstale":"2023-11-28T11:19:00.498382+0000","last_undegraded":"2023-11-28T11:19:00.498382+0000","last_fullsized":"2023-11-28T11:19:00.498382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:27.462121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532313+0000","last_change":"2023-11-28T11:18:59.530295+0000","last_active":"2023-11-28T11:18:59.532313+0000","last_peered":"2023-11-28T11:18:59.532313+0000","last_clean":"2023-11-28T11:18:59.532313+0000","last_became_active":"2023-11-28T11:18:59.530001+0000","last_became_peered":"2023-11-28T11:18:59.530001+0000","last_unstale":"2023-11-28T11:18:59.532313+0000","last_undegraded":"2023-11-28T11:18:59.532313+0000","last_fullsized":"2023-11-28T11:18:59.532313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532745+0000","last_change":"2023-11-28T11:18:59.530935+0000","last_active":"2023-11-28T11:18:59.532745+0000","last_peered":"2023-11-28T11:18:59.532745+0000","last_clean":"2023-11-28T11:18:59.532745+0000","last_became_active":"2023-11-28T11:18:59.530436+0000","last_became_peered":"2023-11-28T11:18:59.530436+0000","last_unstale":"2023-11-28T11:18:59.532745+0000","last_undegraded":"2023-11-28T11:18:59.532745+0000","last_fullsized":"2023-11-28T11:18:59.532745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747752+0000","last_change":"2023-11-28T11:18:59.520394+0000","last_active":"2023-11-28T11:19:00.747752+0000","last_peered":"2023-11-28T11:19:00.747752+0000","last_clean":"2023-11-28T11:19:00.747752+0000","last_became_active":"2023-11-28T11:18:59.519198+0000","last_became_peered":"2023-11-28T11:18:59.519198+0000","last_unstale":"2023-11-28T11:19:00.747752+0000","last_undegraded":"2023-11-28T11:19:00.747752+0000","last_fullsized":"2023-11-28T11:19:00.747752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:53.850423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519945+0000","last_change":"2023-11-28T11:18:59.514188+0000","last_active":"2023-11-28T11:18:59.519945+0000","last_peered":"2023-11-28T11:18:59.519945+0000","last_clean":"2023-11-28T11:18:59.519945+0000","last_became_active":"2023-11-28T11:18:59.513929+0000","last_became_peered":"2023-11-28T11:18:59.513929+0000","last_unstale":"2023-11-28T11:18:59.519945+0000","last_undegraded":"2023-11-28T11:18:59.519945+0000","last_fullsized":"2023-11-28T11:18:59.519945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:35.053943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747636+0000","last_change":"2023-11-28T11:18:59.530152+0000","last_active":"2023-11-28T11:19:00.747636+0000","last_peered":"2023-11-28T11:19:00.747636+0000","last_clean":"2023-11-28T11:19:00.747636+0000","last_became_active":"2023-11-28T11:18:59.528030+0000","last_became_peered":"2023-11-28T11:18:59.528030+0000","last_unstale":"2023-11-28T11:19:00.747636+0000","last_undegraded":"2023-11-28T11:19:00.747636+0000","last_fullsized":"2023-11-28T11:19:00.747636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:16.864464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498030+0000","last_change":"2023-11-28T11:18:59.521195+0000","last_active":"2023-11-28T11:19:00.498030+0000","last_peered":"2023-11-28T11:19:00.498030+0000","last_clean":"2023-11-28T11:19:00.498030+0000","last_became_active":"2023-11-28T11:18:59.515301+0000","last_became_peered":"2023-11-28T11:18:59.515301+0000","last_unstale":"2023-11-28T11:19:00.498030+0000","last_undegraded":"2023-11-28T11:19:00.498030+0000","last_fullsized":"2023-11-28T11:19:00.498030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:56:01.821322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498496+0000","last_change":"2023-11-28T11:18:59.515641+0000","last_active":"2023-11-28T11:19:00.498496+0000","last_peered":"2023-11-28T11:19:00.498496+0000","last_clean":"2023-11-28T11:19:00.498496+0000","last_became_active":"2023-11-28T11:18:59.515343+0000","last_became_peered":"2023-11-28T11:18:59.515343+0000","last_unstale":"2023-11-28T11:19:00.498496+0000","last_undegraded":"2023-11-28T11:19:00.498496+0000","last_fullsized":"2023-11-28T11:19:00.498496+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:04.494294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525309+0000","last_change":"2023-11-28T11:18:59.520502+0000","last_active":"2023-11-28T11:18:59.525309+0000","last_peered":"2023-11-28T11:18:59.525309+0000","last_clean":"2023-11-28T11:18:59.525309+0000","last_became_active":"2023-11-28T11:18:59.519898+0000","last_became_peered":"2023-11-28T11:18:59.519898+0000","last_unstale":"2023-11-28T11:18:59.525309+0000","last_undegraded":"2023-11-28T11:18:59.525309+0000","last_fullsized":"2023-11-28T11:18:59.525309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525436+0000","last_change":"2023-11-28T11:18:59.524078+0000","last_active":"2023-11-28T11:18:59.525436+0000","last_peered":"2023-11-28T11:18:59.525436+0000","last_clean":"2023-11-28T11:18:59.525436+0000","last_became_active":"2023-11-28T11:18:59.523785+0000","last_became_peered":"2023-11-28T11:18:59.523785+0000","last_unstale":"2023-11-28T11:18:59.525436+0000","last_undegraded":"2023-11-28T11:18:59.525436+0000","last_fullsized":"2023-11-28T11:18:59.525436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:38.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747802+0000","last_change":"2023-11-28T11:18:59.520009+0000","last_active":"2023-11-28T11:19:00.747802+0000","last_peered":"2023-11-28T11:19:00.747802+0000","last_clean":"2023-11-28T11:19:00.747802+0000","last_became_active":"2023-11-28T11:18:59.517949+0000","last_became_peered":"2023-11-28T11:18:59.517949+0000","last_unstale":"2023-11-28T11:19:00.747802+0000","last_undegraded":"2023-11-28T11:19:00.747802+0000","last_fullsized":"2023-11-28T11:19:00.747802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:01.731030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497235+0000","last_change":"2023-11-28T11:18:59.528917+0000","last_active":"2023-11-28T11:19:00.497235+0000","last_peered":"2023-11-28T11:19:00.497235+0000","last_clean":"2023-11-28T11:19:00.497235+0000","last_became_active":"2023-11-28T11:18:59.528647+0000","last_became_peered":"2023-11-28T11:18:59.528647+0000","last_unstale":"2023-11-28T11:19:00.497235+0000","last_undegraded":"2023-11-28T11:19:00.497235+0000","last_fullsized":"2023-11-28T11:19:00.497235+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.468149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747627+0000","last_change":"2023-11-28T11:18:59.520290+0000","last_active":"2023-11-28T11:19:00.747627+0000","last_peered":"2023-11-28T11:19:00.747627+0000","last_clean":"2023-11-28T11:19:00.747627+0000","last_became_active":"2023-11-28T11:18:59.519328+0000","last_became_peered":"2023-11-28T11:18:59.519328+0000","last_unstale":"2023-11-28T11:19:00.747627+0000","last_undegraded":"2023-11-28T11:19:00.747627+0000","last_fullsized":"2023-11-28T11:19:00.747627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:27.729983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498935+0000","last_change":"2023-11-28T11:18:59.527402+0000","last_active":"2023-11-28T11:19:00.498935+0000","last_peered":"2023-11-28T11:19:00.498935+0000","last_clean":"2023-11-28T11:19:00.498935+0000","last_became_active":"2023-11-28T11:18:59.525202+0000","last_became_peered":"2023-11-28T11:18:59.525202+0000","last_unstale":"2023-11-28T11:19:00.498935+0000","last_undegraded":"2023-11-28T11:19:00.498935+0000","last_fullsized":"2023-11-28T11:19:00.498935+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:38.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747591+0000","last_change":"2023-11-28T11:18:59.513537+0000","last_active":"2023-11-28T11:19:00.747591+0000","last_peered":"2023-11-28T11:19:00.747591+0000","last_clean":"2023-11-28T11:19:00.747591+0000","last_became_active":"2023-11-28T11:18:59.513253+0000","last_became_peered":"2023-11-28T11:18:59.513253+0000","last_unstale":"2023-11-28T11:19:00.747591+0000","last_undegraded":"2023-11-28T11:19:00.747591+0000","last_fullsized":"2023-11-28T11:19:00.747591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:32.314154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747503+0000","last_change":"2023-11-28T11:18:59.520300+0000","last_active":"2023-11-28T11:19:00.747503+0000","last_peered":"2023-11-28T11:19:00.747503+0000","last_clean":"2023-11-28T11:19:00.747503+0000","last_became_active":"2023-11-28T11:18:59.519955+0000","last_became_peered":"2023-11-28T11:18:59.519955+0000","last_unstale":"2023-11-28T11:19:00.747503+0000","last_undegraded":"2023-11-28T11:19:00.747503+0000","last_fullsized":"2023-11-28T11:19:00.747503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:30.358902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525436+0000","last_change":"2023-11-28T11:18:59.520674+0000","last_active":"2023-11-28T11:18:59.525436+0000","last_peered":"2023-11-28T11:18:59.525436+0000","last_clean":"2023-11-28T11:18:59.525436+0000","last_became_active":"2023-11-28T11:18:59.520251+0000","last_became_peered":"2023-11-28T11:18:59.520251+0000","last_unstale":"2023-11-28T11:18:59.525436+0000","last_undegraded":"2023-11-28T11:18:59.525436+0000","last_fullsized":"2023-11-28T11:18:59.525436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497284+0000","last_change":"2023-11-28T11:18:59.518778+0000","last_active":"2023-11-28T11:19:00.497284+0000","last_peered":"2023-11-28T11:19:00.497284+0000","last_clean":"2023-11-28T11:19:00.497284+0000","last_became_active":"2023-11-28T11:18:59.518535+0000","last_became_peered":"2023-11-28T11:18:59.518535+0000","last_unstale":"2023-11-28T11:19:00.497284+0000","last_undegraded":"2023-11-28T11:19:00.497284+0000","last_fullsized":"2023-11-28T11:19:00.497284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:28.302627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.531686+0000","last_change":"2023-11-28T11:18:59.529081+0000","last_active":"2023-11-28T11:18:59.531686+0000","last_peered":"2023-11-28T11:18:59.531686+0000","last_clean":"2023-11-28T11:18:59.531686+0000","last_became_active":"2023-11-28T11:18:59.528801+0000","last_became_peered":"2023-11-28T11:18:59.528801+0000","last_unstale":"2023-11-28T11:18:59.531686+0000","last_undegraded":"2023-11-28T11:18:59.531686+0000","last_fullsized":"2023-11-28T11:18:59.531686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747861+0000","last_change":"2023-11-28T11:18:58.982271+0000","last_active":"2023-11-28T11:19:00.747861+0000","last_peered":"2023-11-28T11:19:00.747861+0000","last_clean":"2023-11-28T11:19:00.747861+0000","last_became_active":"2023-11-28T11:18:58.981951+0000","last_became_peered":"2023-11-28T11:18:58.981951+0000","last_unstale":"2023-11-28T11:19:00.747861+0000","last_undegraded":"2023-11-28T11:19:00.747861+0000","last_fullsized":"2023-11-28T11:19:00.747861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:35.247644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497386+0000","last_change":"2023-11-28T11:18:59.531050+0000","last_active":"2023-11-28T11:19:00.497386+0000","last_peered":"2023-11-28T11:19:00.497386+0000","last_clean":"2023-11-28T11:19:00.497386+0000","last_became_active":"2023-11-28T11:18:59.529001+0000","last_became_peered":"2023-11-28T11:18:59.529001+0000","last_unstale":"2023-11-28T11:19:00.497386+0000","last_undegraded":"2023-11-28T11:19:00.497386+0000","last_fullsized":"2023-11-28T11:19:00.497386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:53.259503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524539+0000","last_change":"2023-11-28T11:18:58.982012+0000","last_active":"2023-11-28T11:18:59.524539+0000","last_peered":"2023-11-28T11:18:59.524539+0000","last_clean":"2023-11-28T11:18:59.524539+0000","last_became_active":"2023-11-28T11:18:58.980911+0000","last_became_peered":"2023-11-28T11:18:58.980911+0000","last_unstale":"2023-11-28T11:18:59.524539+0000","last_undegraded":"2023-11-28T11:18:59.524539+0000","last_fullsized":"2023-11-28T11:18:59.524539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:24.957041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498189+0000","last_change":"2023-11-28T11:18:59.523567+0000","last_active":"2023-11-28T11:19:00.498189+0000","last_peered":"2023-11-28T11:19:00.498189+0000","last_clean":"2023-11-28T11:19:00.498189+0000","last_became_active":"2023-11-28T11:18:59.520356+0000","last_became_peered":"2023-11-28T11:18:59.520356+0000","last_unstale":"2023-11-28T11:19:00.498189+0000","last_undegraded":"2023-11-28T11:19:00.498189+0000","last_fullsized":"2023-11-28T11:19:00.498189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:07.263691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497529+0000","last_change":"2023-11-28T11:18:58.985409+0000","last_active":"2023-11-28T11:19:00.497529+0000","last_peered":"2023-11-28T11:19:00.497529+0000","last_clean":"2023-11-28T11:19:00.497529+0000","last_became_active":"2023-11-28T11:18:58.983943+0000","last_became_peered":"2023-11-28T11:18:58.983943+0000","last_unstale":"2023-11-28T11:19:00.497529+0000","last_undegraded":"2023-11-28T11:19:00.497529+0000","last_fullsized":"2023-11-28T11:19:00.497529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:31.307853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522173+0000","last_change":"2023-11-28T11:18:59.521266+0000","last_active":"2023-11-28T11:18:59.522173+0000","last_peered":"2023-11-28T11:18:59.522173+0000","last_clean":"2023-11-28T11:18:59.522173+0000","last_became_active":"2023-11-28T11:18:59.519421+0000","last_became_peered":"2023-11-28T11:18:59.519421+0000","last_unstale":"2023-11-28T11:18:59.522173+0000","last_undegraded":"2023-11-28T11:18:59.522173+0000","last_fullsized":"2023-11-28T11:18:59.522173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:02.893986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498237+0000","last_change":"2023-11-28T11:18:58.987642+0000","last_active":"2023-11-28T11:19:00.498237+0000","last_peered":"2023-11-28T11:19:00.498237+0000","last_clean":"2023-11-28T11:19:00.498237+0000","last_became_active":"2023-11-28T11:18:58.987103+0000","last_became_peered":"2023-11-28T11:18:58.987103+0000","last_unstale":"2023-11-28T11:19:00.498237+0000","last_undegraded":"2023-11-28T11:19:00.498237+0000","last_fullsized":"2023-11-28T11:19:00.498237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.385763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.516606+0000","last_change":"2023-11-28T11:18:59.514522+0000","last_active":"2023-11-28T11:18:59.516606+0000","last_peered":"2023-11-28T11:18:59.516606+0000","last_clean":"2023-11-28T11:18:59.516606+0000","last_became_active":"2023-11-28T11:18:59.514262+0000","last_became_peered":"2023-11-28T11:18:59.514262+0000","last_unstale":"2023-11-28T11:18:59.516606+0000","last_undegraded":"2023-11-28T11:18:59.516606+0000","last_fullsized":"2023-11-28T11:18:59.516606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:23.883676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498696+0000","last_change":"2023-11-28T11:18:58.988031+0000","last_active":"2023-11-28T11:19:00.498696+0000","last_peered":"2023-11-28T11:19:00.498696+0000","last_clean":"2023-11-28T11:19:00.498696+0000","last_became_active":"2023-11-28T11:18:58.987733+0000","last_became_peered":"2023-11-28T11:18:58.987733+0000","last_unstale":"2023-11-28T11:19:00.498696+0000","last_undegraded":"2023-11-28T11:19:00.498696+0000","last_fullsized":"2023-11-28T11:19:00.498696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:45.176553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498682+0000","last_change":"2023-11-28T11:18:59.516963+0000","last_active":"2023-11-28T11:19:00.498682+0000","last_peered":"2023-11-28T11:19:00.498682+0000","last_clean":"2023-11-28T11:19:00.498682+0000","last_became_active":"2023-11-28T11:18:59.516573+0000","last_became_peered":"2023-11-28T11:18:59.516573+0000","last_unstale":"2023-11-28T11:19:00.498682+0000","last_undegraded":"2023-11-28T11:19:00.498682+0000","last_fullsized":"2023-11-28T11:19:00.498682+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:54.102035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524318+0000","last_change":"2023-11-28T11:18:58.994960+0000","last_active":"2023-11-28T11:18:59.524318+0000","last_peered":"2023-11-28T11:18:59.524318+0000","last_clean":"2023-11-28T11:18:59.524318+0000","last_became_active":"2023-11-28T11:18:58.994684+0000","last_became_peered":"2023-11-28T11:18:58.994684+0000","last_unstale":"2023-11-28T11:18:59.524318+0000","last_undegraded":"2023-11-28T11:18:59.524318+0000","last_fullsized":"2023-11-28T11:18:59.524318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:00.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498005+0000","last_change":"2023-11-28T11:18:59.516604+0000","last_active":"2023-11-28T11:19:00.498005+0000","last_peered":"2023-11-28T11:19:00.498005+0000","last_clean":"2023-11-28T11:19:00.498005+0000","last_became_active":"2023-11-28T11:18:59.516286+0000","last_became_peered":"2023-11-28T11:18:59.516286+0000","last_unstale":"2023-11-28T11:19:00.498005+0000","last_undegraded":"2023-11-28T11:19:00.498005+0000","last_fullsized":"2023-11-28T11:19:00.498005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:20:49.583622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497253+0000","last_change":"2023-11-28T11:18:58.995233+0000","last_active":"2023-11-28T11:19:00.497253+0000","last_peered":"2023-11-28T11:19:00.497253+0000","last_clean":"2023-11-28T11:19:00.497253+0000","last_became_active":"2023-11-28T11:18:58.994941+0000","last_became_peered":"2023-11-28T11:18:58.994941+0000","last_unstale":"2023-11-28T11:19:00.497253+0000","last_undegraded":"2023-11-28T11:19:00.497253+0000","last_fullsized":"2023-11-28T11:19:00.497253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:29.352602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.513161+0000","last_change":"2023-11-28T11:18:59.504304+0000","last_active":"2023-11-28T11:18:59.513161+0000","last_peered":"2023-11-28T11:18:59.513161+0000","last_clean":"2023-11-28T11:18:59.513161+0000","last_became_active":"2023-11-28T11:18:59.502864+0000","last_became_peered":"2023-11-28T11:18:59.502864+0000","last_unstale":"2023-11-28T11:18:59.513161+0000","last_undegraded":"2023-11-28T11:18:59.513161+0000","last_fullsized":"2023-11-28T11:18:59.513161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:29.741286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497746+0000","last_change":"2023-11-28T11:18:58.989845+0000","last_active":"2023-11-28T11:19:00.497746+0000","last_peered":"2023-11-28T11:19:00.497746+0000","last_clean":"2023-11-28T11:19:00.497746+0000","last_became_active":"2023-11-28T11:18:58.989473+0000","last_became_peered":"2023-11-28T11:18:58.989473+0000","last_unstale":"2023-11-28T11:19:00.497746+0000","last_undegraded":"2023-11-28T11:19:00.497746+0000","last_fullsized":"2023-11-28T11:19:00.497746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:22.877375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.526097+0000","last_change":"2023-11-28T11:18:59.525388+0000","last_active":"2023-11-28T11:18:59.526097+0000","last_peered":"2023-11-28T11:18:59.526097+0000","last_clean":"2023-11-28T11:18:59.526097+0000","last_became_active":"2023-11-28T11:18:59.525136+0000","last_became_peered":"2023-11-28T11:18:59.525136+0000","last_unstale":"2023-11-28T11:18:59.526097+0000","last_undegraded":"2023-11-28T11:18:59.526097+0000","last_fullsized":"2023-11-28T11:18:59.526097+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:01.731030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.495889+0000","last_change":"2023-11-28T11:18:58.989231+0000","last_active":"2023-11-28T11:18:59.495889+0000","last_peered":"2023-11-28T11:18:59.495889+0000","last_clean":"2023-11-28T11:18:59.495889+0000","last_became_active":"2023-11-28T11:18:58.988914+0000","last_became_peered":"2023-11-28T11:18:58.988914+0000","last_unstale":"2023-11-28T11:18:59.495889+0000","last_undegraded":"2023-11-28T11:18:59.495889+0000","last_fullsized":"2023-11-28T11:18:59.495889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:03.333903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.526438+0000","last_change":"2023-11-28T11:18:59.521471+0000","last_active":"2023-11-28T11:18:59.526438+0000","last_peered":"2023-11-28T11:18:59.526438+0000","last_clean":"2023-11-28T11:18:59.526438+0000","last_became_active":"2023-11-28T11:18:59.521210+0000","last_became_peered":"2023-11-28T11:18:59.521210+0000","last_unstale":"2023-11-28T11:18:59.526438+0000","last_undegraded":"2023-11-28T11:18:59.526438+0000","last_fullsized":"2023-11-28T11:18:59.526438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:29.741286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499049+0000","last_change":"2023-11-28T11:18:58.997601+0000","last_active":"2023-11-28T11:19:00.499049+0000","last_peered":"2023-11-28T11:19:00.499049+0000","last_clean":"2023-11-28T11:19:00.499049+0000","last_became_active":"2023-11-28T11:18:58.997116+0000","last_became_peered":"2023-11-28T11:18:58.997116+0000","last_unstale":"2023-11-28T11:19:00.499049+0000","last_undegraded":"2023-11-28T11:19:00.499049+0000","last_fullsized":"2023-11-28T11:19:00.499049+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:06.439357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515402+0000","last_change":"2023-11-28T11:18:59.508337+0000","last_active":"2023-11-28T11:18:59.515402+0000","last_peered":"2023-11-28T11:18:59.515402+0000","last_clean":"2023-11-28T11:18:59.515402+0000","last_became_active":"2023-11-28T11:18:59.508143+0000","last_became_peered":"2023-11-28T11:18:59.508143+0000","last_unstale":"2023-11-28T11:18:59.515402+0000","last_undegraded":"2023-11-28T11:18:59.515402+0000","last_fullsized":"2023-11-28T11:18:59.515402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499237+0000","last_change":"2023-11-28T11:18:58.992564+0000","last_active":"2023-11-28T11:19:00.499237+0000","last_peered":"2023-11-28T11:19:00.499237+0000","last_clean":"2023-11-28T11:19:00.499237+0000","last_became_active":"2023-11-28T11:18:58.992295+0000","last_became_peered":"2023-11-28T11:18:58.992295+0000","last_unstale":"2023-11-28T11:19:00.499237+0000","last_undegraded":"2023-11-28T11:19:00.499237+0000","last_fullsized":"2023-11-28T11:19:00.499237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:41.179640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498484+0000","last_change":"2023-11-28T11:18:59.519390+0000","last_active":"2023-11-28T11:19:00.498484+0000","last_peered":"2023-11-28T11:19:00.498484+0000","last_clean":"2023-11-28T11:19:00.498484+0000","last_became_active":"2023-11-28T11:18:59.519109+0000","last_became_peered":"2023-11-28T11:18:59.519109+0000","last_unstale":"2023-11-28T11:19:00.498484+0000","last_undegraded":"2023-11-28T11:19:00.498484+0000","last_fullsized":"2023-11-28T11:19:00.498484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:18.186373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532999+0000","last_change":"2023-11-28T11:18:58.990113+0000","last_active":"2023-11-28T11:18:59.532999+0000","last_peered":"2023-11-28T11:18:59.532999+0000","last_clean":"2023-11-28T11:18:59.532999+0000","last_became_active":"2023-11-28T11:18:58.989849+0000","last_became_peered":"2023-11-28T11:18:58.989849+0000","last_unstale":"2023-11-28T11:18:59.532999+0000","last_undegraded":"2023-11-28T11:18:59.532999+0000","last_fullsized":"2023-11-28T11:18:59.532999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:34.047643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.532252+0000","last_change":"2023-11-28T11:18:59.529632+0000","last_active":"2023-11-28T11:18:59.532252+0000","last_peered":"2023-11-28T11:18:59.532252+0000","last_clean":"2023-11-28T11:18:59.532252+0000","last_became_active":"2023-11-28T11:18:59.529360+0000","last_became_peered":"2023-11-28T11:18:59.529360+0000","last_unstale":"2023-11-28T11:18:59.532252+0000","last_undegraded":"2023-11-28T11:18:59.532252+0000","last_fullsized":"2023-11-28T11:18:59.532252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.368483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517825+0000","last_change":"2023-11-28T11:18:58.987543+0000","last_active":"2023-11-28T11:18:59.517825+0000","last_peered":"2023-11-28T11:18:59.517825+0000","last_clean":"2023-11-28T11:18:59.517825+0000","last_became_active":"2023-11-28T11:18:58.987071+0000","last_became_peered":"2023-11-28T11:18:58.987071+0000","last_unstale":"2023-11-28T11:18:59.517825+0000","last_undegraded":"2023-11-28T11:18:59.517825+0000","last_fullsized":"2023-11-28T11:18:59.517825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:37.461831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.517583+0000","last_change":"2023-11-28T11:18:59.516217+0000","last_active":"2023-11-28T11:18:59.517583+0000","last_peered":"2023-11-28T11:18:59.517583+0000","last_clean":"2023-11-28T11:18:59.517583+0000","last_became_active":"2023-11-28T11:18:59.516020+0000","last_became_peered":"2023-11-28T11:18:59.516020+0000","last_unstale":"2023-11-28T11:18:59.517583+0000","last_undegraded":"2023-11-28T11:18:59.517583+0000","last_fullsized":"2023-11-28T11:18:59.517583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:30.600361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497102+0000","last_change":"2023-11-28T11:18:58.999366+0000","last_active":"2023-11-28T11:19:00.497102+0000","last_peered":"2023-11-28T11:19:00.497102+0000","last_clean":"2023-11-28T11:19:00.497102+0000","last_became_active":"2023-11-28T11:18:58.999074+0000","last_became_peered":"2023-11-28T11:18:58.999074+0000","last_unstale":"2023-11-28T11:19:00.497102+0000","last_undegraded":"2023-11-28T11:19:00.497102+0000","last_fullsized":"2023-11-28T11:19:00.497102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:52.844122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.524386+0000","last_change":"2023-11-28T11:18:59.518072+0000","last_active":"2023-11-28T11:18:59.524386+0000","last_peered":"2023-11-28T11:18:59.524386+0000","last_clean":"2023-11-28T11:18:59.524386+0000","last_became_active":"2023-11-28T11:18:59.517757+0000","last_became_peered":"2023-11-28T11:18:59.517757+0000","last_unstale":"2023-11-28T11:18:59.524386+0000","last_undegraded":"2023-11-28T11:18:59.524386+0000","last_fullsized":"2023-11-28T11:18:59.524386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:44.392064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747387+0000","last_change":"2023-11-28T11:18:58.988319+0000","last_active":"2023-11-28T11:19:00.747387+0000","last_peered":"2023-11-28T11:19:00.747387+0000","last_clean":"2023-11-28T11:19:00.747387+0000","last_became_active":"2023-11-28T11:18:58.986732+0000","last_became_peered":"2023-11-28T11:18:58.986732+0000","last_unstale":"2023-11-28T11:19:00.747387+0000","last_undegraded":"2023-11-28T11:19:00.747387+0000","last_fullsized":"2023-11-28T11:19:00.747387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:26.328175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.514444+0000","last_change":"2023-11-28T11:18:59.504921+0000","last_active":"2023-11-28T11:18:59.514444+0000","last_peered":"2023-11-28T11:18:59.514444+0000","last_clean":"2023-11-28T11:18:59.514444+0000","last_became_active":"2023-11-28T11:18:59.504639+0000","last_became_peered":"2023-11-28T11:18:59.504639+0000","last_unstale":"2023-11-28T11:18:59.514444+0000","last_undegraded":"2023-11-28T11:18:59.514444+0000","last_fullsized":"2023-11-28T11:18:59.514444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:08:00.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.529710+0000","last_change":"2023-11-28T11:18:58.992215+0000","last_active":"2023-11-28T11:18:59.529710+0000","last_peered":"2023-11-28T11:18:59.529710+0000","last_clean":"2023-11-28T11:18:59.529710+0000","last_became_active":"2023-11-28T11:18:58.991938+0000","last_became_peered":"2023-11-28T11:18:58.991938+0000","last_unstale":"2023-11-28T11:18:59.529710+0000","last_undegraded":"2023-11-28T11:18:59.529710+0000","last_fullsized":"2023-11-28T11:18:59.529710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:29.352602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519060+0000","last_change":"2023-11-28T11:18:59.517000+0000","last_active":"2023-11-28T11:18:59.519060+0000","last_peered":"2023-11-28T11:18:59.519060+0000","last_clean":"2023-11-28T11:18:59.519060+0000","last_became_active":"2023-11-28T11:18:59.515317+0000","last_became_peered":"2023-11-28T11:18:59.515317+0000","last_unstale":"2023-11-28T11:18:59.519060+0000","last_undegraded":"2023-11-28T11:18:59.519060+0000","last_fullsized":"2023-11-28T11:18:59.519060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499484+0000","last_change":"2023-11-28T11:18:58.994494+0000","last_active":"2023-11-28T11:19:00.499484+0000","last_peered":"2023-11-28T11:19:00.499484+0000","last_clean":"2023-11-28T11:19:00.499484+0000","last_became_active":"2023-11-28T11:18:58.993842+0000","last_became_peered":"2023-11-28T11:18:58.993842+0000","last_unstale":"2023-11-28T11:19:00.499484+0000","last_undegraded":"2023-11-28T11:19:00.499484+0000","last_fullsized":"2023-11-28T11:19:00.499484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:42:44.182419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499129+0000","last_change":"2023-11-28T11:18:59.527401+0000","last_active":"2023-11-28T11:19:00.499129+0000","last_peered":"2023-11-28T11:19:00.499129+0000","last_clean":"2023-11-28T11:19:00.499129+0000","last_became_active":"2023-11-28T11:18:59.524390+0000","last_became_peered":"2023-11-28T11:18:59.524390+0000","last_unstale":"2023-11-28T11:19:00.499129+0000","last_undegraded":"2023-11-28T11:19:00.499129+0000","last_fullsized":"2023-11-28T11:19:00.499129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:55.258780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498415+0000","last_change":"2023-11-28T11:18:58.991079+0000","last_active":"2023-11-28T11:19:00.498415+0000","last_peered":"2023-11-28T11:19:00.498415+0000","last_clean":"2023-11-28T11:19:00.498415+0000","last_became_active":"2023-11-28T11:18:58.990803+0000","last_became_peered":"2023-11-28T11:18:58.990803+0000","last_unstale":"2023-11-28T11:19:00.498415+0000","last_undegraded":"2023-11-28T11:19:00.498415+0000","last_fullsized":"2023-11-28T11:19:00.498415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:28.734985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498301+0000","last_change":"2023-11-28T11:18:59.506613+0000","last_active":"2023-11-28T11:19:00.498301+0000","last_peered":"2023-11-28T11:19:00.498301+0000","last_clean":"2023-11-28T11:19:00.498301+0000","last_became_active":"2023-11-28T11:18:59.506258+0000","last_became_peered":"2023-11-28T11:18:59.506258+0000","last_unstale":"2023-11-28T11:19:00.498301+0000","last_undegraded":"2023-11-28T11:19:00.498301+0000","last_fullsized":"2023-11-28T11:19:00.498301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:38.099722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.529893+0000","last_change":"2023-11-28T11:18:58.990358+0000","last_active":"2023-11-28T11:18:59.529893+0000","last_peered":"2023-11-28T11:18:59.529893+0000","last_clean":"2023-11-28T11:18:59.529893+0000","last_became_active":"2023-11-28T11:18:58.990090+0000","last_became_peered":"2023-11-28T11:18:58.990090+0000","last_unstale":"2023-11-28T11:18:59.529893+0000","last_undegraded":"2023-11-28T11:18:59.529893+0000","last_fullsized":"2023-11-28T11:18:59.529893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:20.314008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.521497+0000","last_change":"2023-11-28T11:18:59.519405+0000","last_active":"2023-11-28T11:18:59.521497+0000","last_peered":"2023-11-28T11:18:59.521497+0000","last_clean":"2023-11-28T11:18:59.521497+0000","last_became_active":"2023-11-28T11:18:59.519143+0000","last_became_peered":"2023-11-28T11:18:59.519143+0000","last_unstale":"2023-11-28T11:18:59.521497+0000","last_undegraded":"2023-11-28T11:18:59.521497+0000","last_fullsized":"2023-11-28T11:18:59.521497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:53.672693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497124+0000","last_change":"2023-11-28T11:18:58.984350+0000","last_active":"2023-11-28T11:19:00.497124+0000","last_peered":"2023-11-28T11:19:00.497124+0000","last_clean":"2023-11-28T11:19:00.497124+0000","last_became_active":"2023-11-28T11:18:58.983962+0000","last_became_peered":"2023-11-28T11:18:58.983962+0000","last_unstale":"2023-11-28T11:19:00.497124+0000","last_undegraded":"2023-11-28T11:19:00.497124+0000","last_fullsized":"2023-11-28T11:19:00.497124+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:15.858163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.523362+0000","last_change":"2023-11-28T11:18:59.515035+0000","last_active":"2023-11-28T11:18:59.523362+0000","last_peered":"2023-11-28T11:18:59.523362+0000","last_clean":"2023-11-28T11:18:59.523362+0000","last_became_active":"2023-11-28T11:18:59.514703+0000","last_became_peered":"2023-11-28T11:18:59.514703+0000","last_unstale":"2023-11-28T11:18:59.523362+0000","last_undegraded":"2023-11-28T11:18:59.523362+0000","last_fullsized":"2023-11-28T11:18:59.523362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:08.241705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.519675+0000","last_change":"2023-11-28T11:18:58.981751+0000","last_active":"2023-11-28T11:18:59.519675+0000","last_peered":"2023-11-28T11:18:59.519675+0000","last_clean":"2023-11-28T11:18:59.519675+0000","last_became_active":"2023-11-28T11:18:58.981450+0000","last_became_peered":"2023-11-28T11:18:58.981450+0000","last_unstale":"2023-11-28T11:18:59.519675+0000","last_undegraded":"2023-11-28T11:18:59.519675+0000","last_fullsized":"2023-11-28T11:18:59.519675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:21.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515530+0000","last_change":"2023-11-28T11:18:59.507434+0000","last_active":"2023-11-28T11:18:59.515530+0000","last_peered":"2023-11-28T11:18:59.515530+0000","last_clean":"2023-11-28T11:18:59.515530+0000","last_became_active":"2023-11-28T11:18:59.507089+0000","last_became_peered":"2023-11-28T11:18:59.507089+0000","last_unstale":"2023-11-28T11:18:59.515530+0000","last_undegraded":"2023-11-28T11:18:59.515530+0000","last_fullsized":"2023-11-28T11:18:59.515530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.509773+0000","last_change":"2023-11-28T11:18:58.977878+0000","last_active":"2023-11-28T11:18:59.509773+0000","last_peered":"2023-11-28T11:18:59.509773+0000","last_clean":"2023-11-28T11:18:59.509773+0000","last_became_active":"2023-11-28T11:18:58.977593+0000","last_became_peered":"2023-11-28T11:18:58.977593+0000","last_unstale":"2023-11-28T11:18:59.509773+0000","last_undegraded":"2023-11-28T11:18:59.509773+0000","last_fullsized":"2023-11-28T11:18:59.509773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:36.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498692+0000","last_change":"2023-11-28T11:18:59.528812+0000","last_active":"2023-11-28T11:19:00.498692+0000","last_peered":"2023-11-28T11:19:00.498692+0000","last_clean":"2023-11-28T11:19:00.498692+0000","last_became_active":"2023-11-28T11:18:59.528546+0000","last_became_peered":"2023-11-28T11:18:59.528546+0000","last_unstale":"2023-11-28T11:19:00.498692+0000","last_undegraded":"2023-11-28T11:19:00.498692+0000","last_fullsized":"2023-11-28T11:19:00.498692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:36.838348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.499374+0000","last_change":"2023-11-28T11:18:58.994878+0000","last_active":"2023-11-28T11:18:59.499374+0000","last_peered":"2023-11-28T11:18:59.499374+0000","last_clean":"2023-11-28T11:18:59.499374+0000","last_became_active":"2023-11-28T11:18:58.994618+0000","last_became_peered":"2023-11-28T11:18:58.994618+0000","last_unstale":"2023-11-28T11:18:59.499374+0000","last_undegraded":"2023-11-28T11:18:59.499374+0000","last_fullsized":"2023-11-28T11:18:59.499374+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:03.333903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515831+0000","last_change":"2023-11-28T11:18:59.508311+0000","last_active":"2023-11-28T11:18:59.515831+0000","last_peered":"2023-11-28T11:18:59.515831+0000","last_clean":"2023-11-28T11:18:59.515831+0000","last_became_active":"2023-11-28T11:18:59.508042+0000","last_became_peered":"2023-11-28T11:18:59.508042+0000","last_unstale":"2023-11-28T11:18:59.515831+0000","last_undegraded":"2023-11-28T11:18:59.515831+0000","last_fullsized":"2023-11-28T11:18:59.515831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:46.046722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.509884+0000","last_change":"2023-11-28T11:18:58.981912+0000","last_active":"2023-11-28T11:18:59.509884+0000","last_peered":"2023-11-28T11:18:59.509884+0000","last_clean":"2023-11-28T11:18:59.509884+0000","last_became_active":"2023-11-28T11:18:58.981302+0000","last_became_peered":"2023-11-28T11:18:58.981302+0000","last_unstale":"2023-11-28T11:18:59.509884+0000","last_undegraded":"2023-11-28T11:18:59.509884+0000","last_fullsized":"2023-11-28T11:18:59.509884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:01:20.118399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.515648+0000","last_change":"2023-11-28T11:18:59.509879+0000","last_active":"2023-11-28T11:18:59.515648+0000","last_peered":"2023-11-28T11:18:59.515648+0000","last_clean":"2023-11-28T11:18:59.515648+0000","last_became_active":"2023-11-28T11:18:59.507927+0000","last_became_peered":"2023-11-28T11:18:59.507927+0000","last_unstale":"2023-11-28T11:18:59.515648+0000","last_undegraded":"2023-11-28T11:18:59.515648+0000","last_fullsized":"2023-11-28T11:18:59.515648+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:07.099752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.501978+0000","last_change":"2023-11-28T11:18:58.988242+0000","last_active":"2023-11-28T11:18:59.501978+0000","last_peered":"2023-11-28T11:18:59.501978+0000","last_clean":"2023-11-28T11:18:59.501978+0000","last_became_active":"2023-11-28T11:18:58.987966+0000","last_became_peered":"2023-11-28T11:18:58.987966+0000","last_unstale":"2023-11-28T11:18:59.501978+0000","last_undegraded":"2023-11-28T11:18:59.501978+0000","last_fullsized":"2023-11-28T11:18:59.501978+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:52.666393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.504047+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:18:59.504047+0000","last_peered":"2023-11-28T11:18:59.504047+0000","last_clean":"2023-11-28T11:18:59.504047+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:18:59.504047+0000","last_undegraded":"2023-11-28T11:18:59.504047+0000","last_fullsized":"2023-11-28T11:18:59.504047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.512100+0000","last_change":"2023-11-28T11:18:59.503838+0000","last_active":"2023-11-28T11:18:59.512100+0000","last_peered":"2023-11-28T11:18:59.512100+0000","last_clean":"2023-11-28T11:18:59.512100+0000","last_became_active":"2023-11-28T11:18:59.503552+0000","last_became_peered":"2023-11-28T11:18:59.503552+0000","last_unstale":"2023-11-28T11:18:59.512100+0000","last_undegraded":"2023-11-28T11:18:59.512100+0000","last_fullsized":"2023-11-28T11:18:59.512100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:07.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.507576+0000","last_change":"2023-11-28T11:18:58.984310+0000","last_active":"2023-11-28T11:18:59.507576+0000","last_peered":"2023-11-28T11:18:59.507576+0000","last_clean":"2023-11-28T11:18:59.507576+0000","last_became_active":"2023-11-28T11:18:58.984049+0000","last_became_peered":"2023-11-28T11:18:58.984049+0000","last_unstale":"2023-11-28T11:18:59.507576+0000","last_undegraded":"2023-11-28T11:18:59.507576+0000","last_fullsized":"2023-11-28T11:18:59.507576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:37.461831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.510427+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:18:59.510427+0000","last_peered":"2023-11-28T11:18:59.510427+0000","last_clean":"2023-11-28T11:18:59.510427+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:18:59.510427+0000","last_undegraded":"2023-11-28T11:18:59.510427+0000","last_fullsized":"2023-11-28T11:18:59.510427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525018+0000","last_change":"2023-11-28T11:18:59.516005+0000","last_active":"2023-11-28T11:18:59.525018+0000","last_peered":"2023-11-28T11:18:59.525018+0000","last_clean":"2023-11-28T11:18:59.525018+0000","last_became_active":"2023-11-28T11:18:59.515002+0000","last_became_peered":"2023-11-28T11:18:59.515002+0000","last_unstale":"2023-11-28T11:18:59.525018+0000","last_undegraded":"2023-11-28T11:18:59.525018+0000","last_fullsized":"2023-11-28T11:18:59.525018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:50.802855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.510660+0000","last_change":"2023-11-28T11:18:58.987720+0000","last_active":"2023-11-28T11:18:59.510660+0000","last_peered":"2023-11-28T11:18:59.510660+0000","last_clean":"2023-11-28T11:18:59.510660+0000","last_became_active":"2023-11-28T11:18:58.987490+0000","last_became_peered":"2023-11-28T11:18:58.987490+0000","last_unstale":"2023-11-28T11:18:59.510660+0000","last_undegraded":"2023-11-28T11:18:59.510660+0000","last_fullsized":"2023-11-28T11:18:59.510660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:52.951851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.977131+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:18:58.977131+0000","last_peered":"2023-11-28T11:18:58.977131+0000","last_clean":"2023-11-28T11:18:58.977131+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:18:58.977131+0000","last_undegraded":"2023-11-28T11:18:58.977131+0000","last_fullsized":"2023-11-28T11:18:58.977131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499438+0000","last_change":"2023-11-28T11:18:59.525045+0000","last_active":"2023-11-28T11:19:00.499438+0000","last_peered":"2023-11-28T11:19:00.499438+0000","last_clean":"2023-11-28T11:19:00.499438+0000","last_became_active":"2023-11-28T11:18:59.524541+0000","last_became_peered":"2023-11-28T11:18:59.524541+0000","last_unstale":"2023-11-28T11:19:00.499438+0000","last_undegraded":"2023-11-28T11:19:00.499438+0000","last_fullsized":"2023-11-28T11:19:00.499438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:57.492401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.498893+0000","last_change":"2023-11-28T11:18:58.996898+0000","last_active":"2023-11-28T11:18:59.498893+0000","last_peered":"2023-11-28T11:18:59.498893+0000","last_clean":"2023-11-28T11:18:59.498893+0000","last_became_active":"2023-11-28T11:18:58.996609+0000","last_became_peered":"2023-11-28T11:18:58.996609+0000","last_unstale":"2023-11-28T11:18:59.498893+0000","last_undegraded":"2023-11-28T11:18:59.498893+0000","last_fullsized":"2023-11-28T11:18:59.498893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:27.115435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.513787+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:18:59.513787+0000","last_peered":"2023-11-28T11:18:59.513787+0000","last_clean":"2023-11-28T11:18:59.513787+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:18:59.513787+0000","last_undegraded":"2023-11-28T11:18:59.513787+0000","last_fullsized":"2023-11-28T11:18:59.513787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.984716+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:18:58.984716+0000","last_peered":"2023-11-28T11:18:58.984716+0000","last_clean":"2023-11-28T11:18:58.984716+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:18:58.984716+0000","last_undegraded":"2023-11-28T11:18:58.984716+0000","last_fullsized":"2023-11-28T11:18:58.984716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.497241+0000","last_change":"2023-11-28T11:18:59.529816+0000","last_active":"2023-11-28T11:19:00.497241+0000","last_peered":"2023-11-28T11:19:00.497241+0000","last_clean":"2023-11-28T11:19:00.497241+0000","last_became_active":"2023-11-28T11:18:59.529201+0000","last_became_peered":"2023-11-28T11:18:59.529201+0000","last_unstale":"2023-11-28T11:19:00.497241+0000","last_undegraded":"2023-11-28T11:19:00.497241+0000","last_fullsized":"2023-11-28T11:19:00.497241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:07.099752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.499495+0000","last_change":"2023-11-28T11:18:58.998296+0000","last_active":"2023-11-28T11:19:00.499495+0000","last_peered":"2023-11-28T11:19:00.499495+0000","last_clean":"2023-11-28T11:19:00.499495+0000","last_became_active":"2023-11-28T11:18:58.997905+0000","last_became_peered":"2023-11-28T11:18:58.997905+0000","last_unstale":"2023-11-28T11:19:00.499495+0000","last_undegraded":"2023-11-28T11:19:00.499495+0000","last_fullsized":"2023-11-28T11:19:00.499495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:29.967341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.985447+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:18:58.985447+0000","last_peered":"2023-11-28T11:18:58.985447+0000","last_clean":"2023-11-28T11:18:58.985447+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:18:58.985447+0000","last_undegraded":"2023-11-28T11:18:58.985447+0000","last_fullsized":"2023-11-28T11:18:58.985447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.496962+0000","last_change":"2023-11-28T11:18:59.528417+0000","last_active":"2023-11-28T11:19:00.496962+0000","last_peered":"2023-11-28T11:19:00.496962+0000","last_clean":"2023-11-28T11:19:00.496962+0000","last_became_active":"2023-11-28T11:18:59.528158+0000","last_became_peered":"2023-11-28T11:18:59.528158+0000","last_unstale":"2023-11-28T11:19:00.496962+0000","last_undegraded":"2023-11-28T11:19:00.496962+0000","last_fullsized":"2023-11-28T11:19:00.496962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:40.829763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.747504+0000","last_change":"2023-11-28T11:18:58.985101+0000","last_active":"2023-11-28T11:19:00.747504+0000","last_peered":"2023-11-28T11:19:00.747504+0000","last_clean":"2023-11-28T11:19:00.747504+0000","last_became_active":"2023-11-28T11:18:58.984641+0000","last_became_peered":"2023-11-28T11:18:58.984641+0000","last_unstale":"2023-11-28T11:19:00.747504+0000","last_undegraded":"2023-11-28T11:19:00.747504+0000","last_fullsized":"2023-11-28T11:19:00.747504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:31.036152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T11:18:58.993331+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:18:58.993331+0000","last_peered":"2023-11-28T11:18:58.993331+0000","last_clean":"2023-11-28T11:18:58.993331+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:18:58.993331+0000","last_undegraded":"2023-11-28T11:18:58.993331+0000","last_fullsized":"2023-11-28T11:18:58.993331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498131+0000","last_change":"2023-11-28T11:18:59.503325+0000","last_active":"2023-11-28T11:19:00.498131+0000","last_peered":"2023-11-28T11:19:00.498131+0000","last_clean":"2023-11-28T11:19:00.498131+0000","last_became_active":"2023-11-28T11:18:59.502962+0000","last_became_peered":"2023-11-28T11:18:59.502962+0000","last_unstale":"2023-11-28T11:19:00.498131+0000","last_undegraded":"2023-11-28T11:19:00.498131+0000","last_fullsized":"2023-11-28T11:19:00.498131+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:32.721152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:19:00.498406+0000","last_change":"2023-11-28T11:18:58.985530+0000","last_active":"2023-11-28T11:19:00.498406+0000","last_peered":"2023-11-28T11:19:00.498406+0000","last_clean":"2023-11-28T11:19:00.498406+0000","last_became_active":"2023-11-28T11:18:58.985204+0000","last_became_peered":"2023-11-28T11:18:58.985204+0000","last_unstale":"2023-11-28T11:19:00.498406+0000","last_undegraded":"2023-11-28T11:19:00.498406+0000","last_fullsized":"2023-11-28T11:19:00.498406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:49.796554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.503281+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:18:59.503281+0000","last_peered":"2023-11-28T11:18:59.503281+0000","last_clean":"2023-11-28T11:18:59.503281+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:18:59.503281+0000","last_undegraded":"2023-11-28T11:18:59.503281+0000","last_fullsized":"2023-11-28T11:18:59.503281+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.522467+0000","last_change":"2023-11-28T11:18:59.514225+0000","last_active":"2023-11-28T11:18:59.522467+0000","last_peered":"2023-11-28T11:18:59.522467+0000","last_clean":"2023-11-28T11:18:59.522467+0000","last_became_active":"2023-11-28T11:18:59.513872+0000","last_became_peered":"2023-11-28T11:18:59.513872+0000","last_unstale":"2023-11-28T11:18:59.522467+0000","last_undegraded":"2023-11-28T11:18:59.522467+0000","last_fullsized":"2023-11-28T11:18:59.522467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:58.961653+0000","last_clean_scrub_stamp":"2023-11-28T11:18:58.961653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.368483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.494843+0000","last_change":"2023-11-28T11:18:58.990718+0000","last_active":"2023-11-28T11:18:59.494843+0000","last_peered":"2023-11-28T11:18:59.494843+0000","last_clean":"2023-11-28T11:18:59.494843+0000","last_became_active":"2023-11-28T11:18:58.990168+0000","last_became_peered":"2023-11-28T11:18:58.990168+0000","last_unstale":"2023-11-28T11:18:59.494843+0000","last_undegraded":"2023-11-28T11:18:59.494843+0000","last_fullsized":"2023-11-28T11:18:59.494843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:18:57.955352+0000","last_clean_scrub_stamp":"2023-11-28T11:18:57.955352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:27.115435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:18:59.525692+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:18:59.525692+0000","last_peered":"2023-11-28T11:18:59.525692+0000","last_clean":"2023-11-28T11:18:59.525692+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:18:59.525692+0000","last_undegraded":"2023-11-28T11:18:59.525692+0000","last_fullsized":"2023-11-28T11:18:59.525692+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13660,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358180,"statfs":{"total":96636764160,"available":96622776320,"internally_reserved":0,"allocated":1003520,"data_stored":753902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77900000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71499999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12440,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359400,"statfs":{"total":96636764160,"available":96624025600,"internally_reserved":0,"allocated":409600,"data_stored":169135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.76100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.1669999999999998,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.81599999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.95999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67200000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12564,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359276,"statfs":{"total":96636764160,"available":96623898624,"internally_reserved":0,"allocated":405504,"data_stored":167695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.60299999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13580,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94358260,"statfs":{"total":96636764160,"available":96622858240,"internally_reserved":0,"allocated":987136,"data_stored":751299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.50900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67400000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12428,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359412,"statfs":{"total":96636764160,"available":96624037888,"internally_reserved":0,"allocated":397312,"data_stored":161986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.76600000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.4580000000000002,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.622},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.69799999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12632,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":12224,"kb_avail":94359208,"statfs":{"total":96636764160,"available":96623828992,"internally_reserved":0,"allocated":409600,"data_stored":163498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12517376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.9929999999999999,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.754}]},{"osd":7,"last update":"Tue Nov 28 11:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.623}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12700,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359140,"statfs":{"total":96636764160,"available":96623759360,"internally_reserved":0,"allocated":413696,"data_stored":169157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.85099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 11:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12572,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359268,"statfs":{"total":96636764160,"available":96623890432,"internally_reserved":0,"allocated":413696,"data_stored":168681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.71699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.63}]},{"osd":2,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.6800000000000002,"5min":2.6800000000000002,"15min":2.6800000000000002},"last":0.64700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:19:04.423 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:19:04.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:19:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.664+0000 7f793f1ff700 1 Processor -- start 2023-11-28T11:19:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.664+0000 7f793f1ff700 1 -- start start 2023-11-28T11:19:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f793f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f793815c9b0 0x7f79380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f793f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7938161340 0x7f79380d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f793f1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79380d1950 con 0x7f79380d2320 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f793f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380b5800 con 0x7f7938161340 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f793f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380b5520 con 0x7f793815c9b0 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f7937fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7938161340 0x7f79380d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f7937fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7938161340 0x7f79380d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50570/0 (socket says 172.21.15.53:50570) 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.665+0000 7f7937fff700 1 -- 172.21.15.53:0/2908872963 learned_addr learned my addr 172.21.15.53:0/2908872963 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f793cf9b700 1 --2- 172.21.15.53:0/2908872963 >> v2:172.21.15.157:3300/0 conn(0x7f793815c9b0 0x7f79380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f7937fff700 1 -- 172.21.15.53:0/2908872963 >> v2:172.21.15.157:3300/0 conn(0x7f793815c9b0 msgr2=0x7f79380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f7937fff700 1 --2- 172.21.15.53:0/2908872963 >> v2:172.21.15.157:3300/0 conn(0x7f793815c9b0 0x7f79380b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f7937fff700 1 -- 172.21.15.53:0/2908872963 >> v1:172.21.15.157:6789/0 conn(0x7f79380d2320 legacy=0x7f79380cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:04.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f7937fff700 1 -- 172.21.15.53:0/2908872963 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380d1300 con 0x7f7938161340 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f7937fff700 1 --2- 172.21.15.53:0/2908872963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7938161340 0x7f79380d1de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7938067460 tx=0x7f7924009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48abf8ca02069988 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.666+0000 7f79377fe700 1 -- 172.21.15.53:0/2908872963 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400b870 con 0x7f7938161340 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f79377fe700 1 -- 172.21.15.53:0/2908872963 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792400b9d0 con 0x7f7938161340 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f79377fe700 1 -- 172.21.15.53:0/2908872963 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400bd60 con 0x7f7938161340 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 -- 172.21.15.53:0/2908872963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7938161340 msgr2=0x7f79380d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2908872963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7938161340 0x7f79380d1de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7938067460 tx=0x7f7924009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 -- 172.21.15.53:0/2908872963 shutdown_connections 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2908872963 >> v2:172.21.15.157:3300/0 conn(0x7f793815c9b0 0x7f79380b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2908872963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7938161340 0x7f79380d1de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 -- 172.21.15.53:0/2908872963 >> 172.21.15.53:0/2908872963 conn(0x7f7938072600 msgr2=0x7f79380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 -- 172.21.15.53:0/2908872963 shutdown_connections 2023-11-28T11:19:04.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.667+0000 7f793f1ff700 1 -- 172.21.15.53:0/2908872963 wait complete. 2023-11-28T11:19:04.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.669+0000 7f793f1ff700 1 Processor -- start 2023-11-28T11:19:04.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.669+0000 7f793f1ff700 1 -- start start 2023-11-28T11:19:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79380d2320 0x7f79381468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 0x7f7938159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c8c80 con 0x7f793815c9b0 2023-11-28T11:19:04.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c8f60 con 0x7f7938161340 2023-11-28T11:19:04.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f793f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c9240 con 0x7f79380d2320 2023-11-28T11:19:04.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f7937fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f7937fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50572/0 (socket says 172.21.15.53:50572) 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.670+0000 7f7937fff700 1 -- 172.21.15.53:0/2502290904 learned_addr learned my addr 172.21.15.53:0/2502290904 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f793cf9b700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79380d2320 0x7f79381468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f793d79c700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 0x7f7938159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 -- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79380d2320 msgr2=0x7f79381468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79380d2320 0x7f79381468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 -- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 msgr2=0x7f7938159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 0x7f7938159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 -- 172.21.15.53:0/2502290904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924009550 con 0x7f793815c9b0 2023-11-28T11:19:04.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f793d79c700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 0x7f7938159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f7937fff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f7924004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924004540 con 0x7f793815c9b0 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240036a0 con 0x7f793815c9b0 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.671+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924003a10 con 0x7f793815c9b0 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.672+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79380b5130 con 0x7f793815c9b0 2023-11-28T11:19:04.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.672+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79380d15e0 con 0x7f793815c9b0 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.673+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79380b52c0 con 0x7f793815c9b0 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.674+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f792400f050 con 0x7f793815c9b0 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.675+0000 7f79357fa700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 0x7f791c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.675+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7924087500 con 0x7f793815c9b0 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.675+0000 7f793cf9b700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 0x7f791c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:04.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.676+0000 7f793cf9b700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 0x7f791c06a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f792c006b20 tx=0x7f792c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:04.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.679+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7924053e10 con 0x7f793815c9b0 2023-11-28T11:19:04.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.964+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f79380745c0 con 0x7f793815c9b0 2023-11-28T11:19:04.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.968+0000 7f79357fa700 1 -- 172.21.15.53:0/2502290904 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7924057430 con 0x7f793815c9b0 2023-11-28T11:19:04.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 msgr2=0x7f791c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 0x7f791c06a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f792c006b20 tx=0x7f792c008040 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 msgr2=0x7f79380c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f7924004750 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 shutdown_connections 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f791c0682f0 0x7f791c06a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79380d2320 0x7f79381468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7938161340 0x7f7938159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 --2- 172.21.15.53:0/2502290904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f793815c9b0 0x7f79380c8230 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 >> 172.21.15.53:0/2502290904 conn(0x7f7938072600 msgr2=0x7f793815d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:04.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 shutdown_connections 2023-11-28T11:19:04.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:04.971+0000 7f793f1ff700 1 -- 172.21.15.53:0/2502290904 wait complete. 2023-11-28T11:19:04.975 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-11-28T11:19:04.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:05.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.192+0000 7f7ae0fff700 1 Processor -- start 2023-11-28T11:19:05.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.193+0000 7f7ae0fff700 1 -- start start 2023-11-28T11:19:05.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ae0fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7adc161340 0x7f7adc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ae0fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc0d2320 0x7f7adc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ae0fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc0d1950 con 0x7f7adc15c9b0 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ae0fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0b5800 con 0x7f7adc161340 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ae0fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b5300 con 0x7f7adc0d2320 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7adc161340 0x7f7adc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7adc161340 0x7f7adc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50584/0 (socket says 172.21.15.53:50584) 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 -- 172.21.15.53:0/23234554 learned_addr learned my addr 172.21.15.53:0/23234554 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7adad9d700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc0d2320 0x7f7adc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 -- 172.21.15.53:0/23234554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc0d2320 msgr2=0x7f7adc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc0d2320 0x7f7adc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 -- 172.21.15.53:0/23234554 >> v1:172.21.15.157:6789/0 conn(0x7f7adc15c9b0 legacy=0x7f7adc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.194+0000 7f7ad9d9b700 1 -- 172.21.15.53:0/23234554 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0b5520 con 0x7f7adc161340 2023-11-28T11:19:05.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ad9d9b700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc161340 0x7f7adc0d1de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098f0 tx=0x7f7ad0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1e5dd32acbbeba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:05.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ad959a700 1 -- 172.21.15.53:0/23234554 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0011830 con 0x7f7adc161340 2023-11-28T11:19:05.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ad959a700 1 -- 172.21.15.53:0/23234554 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0011990 con 0x7f7adc161340 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ad959a700 1 -- 172.21.15.53:0/23234554 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0011d20 con 0x7f7adc161340 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ae0fff700 1 -- 172.21.15.53:0/23234554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc161340 msgr2=0x7f7adc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.195+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc161340 0x7f7adc0d1de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098f0 tx=0x7f7ad0009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 -- 172.21.15.53:0/23234554 shutdown_connections 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc0d2320 0x7f7adc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/23234554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc161340 0x7f7adc0d1de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 -- 172.21.15.53:0/23234554 >> 172.21.15.53:0/23234554 conn(0x7f7adc072600 msgr2=0x7f7adc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 -- 172.21.15.53:0/23234554 shutdown_connections 2023-11-28T11:19:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.196+0000 7f7ae0fff700 1 -- 172.21.15.53:0/23234554 wait complete. 2023-11-28T11:19:05.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.198+0000 7f7ae0fff700 1 Processor -- start 2023-11-28T11:19:05.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.198+0000 7f7ae0fff700 1 -- start start 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc15c9b0 0x7f7adc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc161340 0x7f7adc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1509d0 con 0x7f7adc0d2320 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1e8920 con 0x7f7adc161340 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ae0fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1e79c0 con 0x7f7adc15c9b0 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50600/0 (socket says 172.21.15.53:50600) 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 -- 172.21.15.53:0/756298063 learned_addr learned my addr 172.21.15.53:0/756298063 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ad9d9b700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc15c9b0 0x7f7adc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7adad9d700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc161340 0x7f7adc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 -- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc15c9b0 msgr2=0x7f7adc1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc15c9b0 0x7f7adc1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 -- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc161340 msgr2=0x7f7adc1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc161340 0x7f7adc1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.199+0000 7f7ada59c700 1 -- 172.21.15.53:0/756298063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0009550 con 0x7f7adc0d2320 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7ada59c700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c450 tx=0x7f7acc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0052d0 con 0x7f7adc0d2320 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc005430 con 0x7f7adc0d2320 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0038c0 con 0x7f7adc0d2320 2023-11-28T11:19:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc0d15e0 con 0x7f7adc0d2320 2023-11-28T11:19:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.200+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc0d1300 con 0x7f7adc0d2320 2023-11-28T11:19:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.201+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc15cd80 con 0x7f7adc0d2320 2023-11-28T11:19:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.202+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7acc019030 con 0x7f7adc0d2320 2023-11-28T11:19:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.203+0000 7f7acaffd700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.203+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7acc084e30 con 0x7f7adc0d2320 2023-11-28T11:19:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.203+0000 7f7ad9d9b700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.204+0000 7f7ad9d9b700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 0x7f7abc06a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009c40 tx=0x7f7ad0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.207+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7acc051690 con 0x7f7adc0d2320 2023-11-28T11:19:05.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.455+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7adc0745c0 con 0x7f7adc0d2320 2023-11-28T11:19:05.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.456+0000 7f7acaffd700 1 -- 172.21.15.53:0/756298063 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f7acc017090 con 0x7f7adc0d2320 2023-11-28T11:19:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:05.458 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:05.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 msgr2=0x7f7abc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 0x7f7abc06a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009c40 tx=0x7f7ad0008030 comp rx=0 tx=0).stop 2023-11-28T11:19:05.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 msgr2=0x7f7adc150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:05.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c450 tx=0x7f7acc00c760 comp rx=0 tx=0).stop 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 shutdown_connections 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc161340 0x7f7adc1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7adc0d2320 0x7f7adc150490 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 --2- 172.21.15.53:0/756298063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc15c9b0 0x7f7adc1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 >> 172.21.15.53:0/756298063 conn(0x7f7adc072600 msgr2=0x7f7adc0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 shutdown_connections 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:05.458+0000 7f7ae0fff700 1 -- 172.21.15.53:0/756298063 wait complete. 2023-11-28T11:19:05.461 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:05.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:18:59.490713+0000', 'modified': '2023-11-28T11:19:02.510023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54722}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54722': {'gid': 54722, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6834/3764823575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 3764823575}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 3764823575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:19:05.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:19:05.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:19:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.085+0000 7fd51d1ff700 1 Processor -- start 2023-11-28T11:19:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.085+0000 7fd51d1ff700 1 -- start start 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51d1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 0x7fd5180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51d1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd518161340 0x7fd5180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51d1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180d1950 con 0x7fd5180d2320 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51d1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180b5800 con 0x7fd518161340 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51d1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180b5520 con 0x7fd51815c9b0 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51759e700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd5180d2320 0x7fd5180cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55154/0 (socket says 172.21.15.53:55154) 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd51759e700 1 -- 172.21.15.53:0/867604059 learned_addr learned my addr 172.21.15.53:0/867604059 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd50ffff700 1 --2- 172.21.15.53:0/867604059 >> v2:172.21.15.53:3300/0 conn(0x7fd518161340 0x7fd5180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.087+0000 7fd516d9d700 1 --2- 172.21.15.53:0/867604059 >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 0x7fd5180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd50ffff700 1 -- 172.21.15.53:0/867604059 >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 msgr2=0x7fd5180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd50ffff700 1 --2- 172.21.15.53:0/867604059 >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 0x7fd5180b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd50ffff700 1 -- 172.21.15.53:0/867604059 >> v1:172.21.15.53:6789/0 conn(0x7fd5180d2320 legacy=0x7fd5180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd50ffff700 1 -- 172.21.15.53:0/867604059 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180d1300 con 0x7fd518161340 2023-11-28T11:19:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd516d9d700 1 --2- 172.21.15.53:0/867604059 >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 0x7fd5180b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd51659c700 1 -- 172.21.15.53:0/867604059 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453225227 0 0) 0x7fd5180d1950 con 0x7fd5180d2320 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd50ffff700 1 --2- 172.21.15.53:0/867604059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd518161340 0x7fd5180d1de0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd518067460 tx=0x7fd500009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b54af359c60dee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd51659c700 1 -- 172.21.15.53:0/867604059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500011830 con 0x7fd518161340 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd51659c700 1 -- 172.21.15.53:0/867604059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd500011990 con 0x7fd518161340 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.088+0000 7fd51659c700 1 -- 172.21.15.53:0/867604059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500011d20 con 0x7fd518161340 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 -- 172.21.15.53:0/867604059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd518161340 msgr2=0x7fd5180d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/867604059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd518161340 0x7fd5180d1de0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd518067460 tx=0x7fd500009b20 comp rx=0 tx=0).stop 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 -- 172.21.15.53:0/867604059 shutdown_connections 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/867604059 >> v2:172.21.15.157:3300/0 conn(0x7fd51815c9b0 0x7fd5180b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/867604059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd518161340 0x7fd5180d1de0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 -- 172.21.15.53:0/867604059 >> 172.21.15.53:0/867604059 conn(0x7fd518072600 msgr2=0x7fd5180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 -- 172.21.15.53:0/867604059 shutdown_connections 2023-11-28T11:19:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.089+0000 7fd51d1ff700 1 -- 172.21.15.53:0/867604059 wait complete. 2023-11-28T11:19:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.091+0000 7fd51d1ff700 1 Processor -- start 2023-11-28T11:19:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.091+0000 7fd51d1ff700 1 -- start start 2023-11-28T11:19:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51815c9b0 0x7fd5181e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 0x7fd5180b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518143a50 con 0x7fd5180d2320 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5181554a0 con 0x7fd51815c9b0 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51d1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd518155740 con 0x7fd518161340 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50622/0 (socket says 172.21.15.53:50622) 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 -- 172.21.15.53:0/477328390 learned_addr learned my addr 172.21.15.53:0/477328390 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd50ffff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51815c9b0 0x7fd5181e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51759e700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 0x7fd5180b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 -- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 msgr2=0x7fd5180b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 0x7fd5180b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 -- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51815c9b0 msgr2=0x7fd5181e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51815c9b0 0x7fd5181e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd516d9d700 1 -- 172.21.15.53:0/477328390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000094a0 con 0x7fd5180d2320 2023-11-28T11:19:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.092+0000 7fd51759e700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 0x7fd5180b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd516d9d700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e810 tx=0x7fd50800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd50f7fe700 1 -- 172.21.15.53:0/477328390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508013610 con 0x7fd5180d2320 2023-11-28T11:19:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd50f7fe700 1 -- 172.21.15.53:0/477328390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508013770 con 0x7fd5180d2320 2023-11-28T11:19:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd50f7fe700 1 -- 172.21.15.53:0/477328390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508013b20 con 0x7fd5180d2320 2023-11-28T11:19:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180b5130 con 0x7fd5180d2320 2023-11-28T11:19:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.093+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5180d15e0 con 0x7fd5180d2320 2023-11-28T11:19:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.095+0000 7fd50f7fe700 1 -- 172.21.15.53:0/477328390 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd508023070 con 0x7fd5180d2320 2023-11-28T11:19:06.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.096+0000 7fd50f7fe700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 0x7fd4f806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.096+0000 7fd50f7fe700 1 -- 172.21.15.53:0/477328390 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd508085940 con 0x7fd5180d2320 2023-11-28T11:19:06.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.097+0000 7fd50ffff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 0x7fd4f806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.097+0000 7fd50ffff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 0x7fd4f806a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5000090b0 tx=0x7fd500008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.098+0000 7fd530bb3b80 1 Processor -- start 2023-11-28T11:19:06.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.098+0000 7fd530bb3b80 1 -- start start 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ad810 0x5644e8883a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e876dbc0 con 0x5644e88ab130 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e87dc350 con 0x5644e88ae120 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e87d5880 con 0x5644e88ad810 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50624/0 (socket says 172.21.15.53:50624) 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd50ffff700 1 -- 172.21.15.53:0/844393470 learned_addr learned my addr 172.21.15.53:0/844393470 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd516d9d700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ad810 0x5644e8883a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.099+0000 7fd51759e700 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 -- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ad810 msgr2=0x5644e8883a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ad810 0x5644e8883a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 -- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 msgr2=0x5644e88c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 -- 172.21.15.53:0/844393470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e86575e0 con 0x5644e88ab130 2023-11-28T11:19:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50ffff700 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5180675d0 tx=0x7fd500010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c147594d6170397 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.100+0000 7fd50cff9700 1 -- 172.21.15.53:0/844393470 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50002c070 con 0x5644e88ab130 2023-11-28T11:19:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd50cff9700 1 -- 172.21.15.53:0/844393470 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5000036a0 con 0x5644e88ab130 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd50cff9700 1 -- 172.21.15.53:0/844393470 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500020070 con 0x5644e88ab130 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 -- 172.21.15.53:0/844393470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 msgr2=0x5644e88adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5180675d0 tx=0x7fd500010000 comp rx=0 tx=0).stop 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 -- 172.21.15.53:0/844393470 shutdown_connections 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ad810 0x5644e8883a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/844393470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ab130 0x5644e88adbe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 -- 172.21.15.53:0/844393470 >> 172.21.15.53:0/844393470 conn(0x5644e8880d60 msgr2=0x5644e88816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 -- 172.21.15.53:0/844393470 shutdown_connections 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.101+0000 7fd530bb3b80 1 -- 172.21.15.53:0/844393470 wait complete. 2023-11-28T11:19:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.102+0000 7fd530bb3b80 1 -- start start 2023-11-28T11:19:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.102+0000 7fd530bb3b80 1 Processor -- start 2023-11-28T11:19:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 0x5644e88cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e86598d0 con 0x5644e88ae120 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e88af960 con 0x5644e88ad810 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e88a6040 con 0x5644e88ab130 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.103+0000 7fd530bb3b80 10 client.0 _ll_register_callbacks cb 0x5644e867b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50638/0 (socket says 172.21.15.53:50638) 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 -- 172.21.15.53:0/1365969191 learned_addr learned my addr 172.21.15.53:0/1365969191 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd516d9d700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd50ffff700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 0x5644e88cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 -- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 msgr2=0x5644e88cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 -- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 msgr2=0x5644e88cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 0x5644e88cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 -- 172.21.15.53:0/1365969191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5180d1300 con 0x5644e88ae120 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd50ffff700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 0x5644e88cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd51759e700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd510007b30 tx=0x7fd510011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2703969e1d448ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.104+0000 7fd4f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510011d50 con 0x5644e88ae120 2023-11-28T11:19:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd510015470 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e87a3140 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd510011eb0 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 10 client.5259 fetch_fsmap learned FSMap version 16 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5644e8603d90 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5644e87a3140 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd510015630 con 0x5644e88ae120 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 10 client.5259 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 10 client.5259 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 20 client.5259 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88ce820 0x5644e88cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 4 client.5259 mds_command: new command op to 54722 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:19:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.105+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 --> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644e8659f10 con 0x5644e88ce820 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.106+0000 7fd516d9d700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88ce820 0x5644e88cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.106+0000 7fd4f5ffb700 10 client.5259 ms_handle_connect on v2:172.21.15.53:6833/3764823575 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.106+0000 7fd516d9d700 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88ce820 0x5644e88cebf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9f7b06c0119ae5 server_cookie=75b648d6cdf8d423 in_seq=0 out_seq=0 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/1365969191 <== mds.0 v2:172.21.15.53:6833/3764823575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5644e8659f10 con 0x5644e88ce820 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd4f5ffb700 10 client.5259 handle_command_reply: tid=0 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 client.5259 shutdown 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 msgr2=0x5644e88ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd510007b30 tx=0x7fd510011530 comp rx=0 tx=0).stop 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 shutdown_connections 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88ce820 0x5644e88cebf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ae120 0x5644e88ae9d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1365969191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ad810 0x5644e88cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 >> 172.21.15.53:0/1365969191 conn(0x5644e8880d60 msgr2=0x5644e8883990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd4f5ffb700 0 client.5259 ms_handle_reset on v2:172.21.15.53:6833/3764823575 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 shutdown_connections 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.107+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1365969191 wait complete. 2023-11-28T11:19:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.108+0000 7fd530bb3b80 20 client.5259 trim_cache size 0 max 16384 2023-11-28T11:19:06.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 Processor -- start 2023-11-28T11:19:06.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 -- start start 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e88b0ad0 con 0x5644e88ad810 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e88d0280 con 0x5644e88ae120 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.126+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e887eb80 con 0x5644e88ab130 2023-11-28T11:19:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50650/0 (socket says 172.21.15.53:50650) 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 -- 172.21.15.53:0/1972331789 learned_addr learned my addr 172.21.15.53:0/1972331789 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd51759e700 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd516d9d700 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 -- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 msgr2=0x5644e88a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 -- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 msgr2=0x5644e88affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 -- 172.21.15.53:0/1972331789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e879c140 con 0x5644e88ad810 2023-11-28T11:19:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.127+0000 7fd50ffff700 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd500034040 tx=0x7fd500020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad7001b273632c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd50d7fa700 1 -- 172.21.15.53:0/1972331789 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50002c040 con 0x5644e88ad810 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd50d7fa700 1 -- 172.21.15.53:0/1972331789 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5000038a0 con 0x5644e88ad810 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd50d7fa700 1 -- 172.21.15.53:0/1972331789 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500003c10 con 0x5644e88ad810 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1972331789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 msgr2=0x5644e88afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd500034040 tx=0x7fd500020000 comp rx=0 tx=0).stop 2023-11-28T11:19:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1972331789 shutdown_connections 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ab130 0x5644e88a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ae120 0x5644e88affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/1972331789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e88afaa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1972331789 >> 172.21.15.53:0/1972331789 conn(0x5644e8880d60 msgr2=0x5644e8884a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.128+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1972331789 shutdown_connections 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.129+0000 7fd530bb3b80 1 -- 172.21.15.53:0/1972331789 wait complete. 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.129+0000 7fd530bb3b80 1 -- start start 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.129+0000 7fd530bb3b80 1 Processor -- start 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ab130 0x5644e892d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ae120 0x5644e88e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8884cb0 con 0x5644e88ad810 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e88cc430 con 0x5644e88ab130 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e892eae0 con 0x5644e88ae120 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.130+0000 7fd530bb3b80 10 client.0 _ll_register_callbacks cb 0x7fd5100073f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50664/0 (socket says 172.21.15.53:50664) 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 -- 172.21.15.53:0/902366140 learned_addr learned my addr 172.21.15.53:0/902366140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd51759e700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ae120 0x5644e88e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd516d9d700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ab130 0x5644e892d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 -- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ae120 msgr2=0x5644e88e1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ae120 0x5644e88e1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 -- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ab130 msgr2=0x5644e892d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ab130 0x5644e892d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 -- 172.21.15.53:0/902366140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644e86575e0 con 0x5644e88ad810 2023-11-28T11:19:06.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd50ffff700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd500009f20 tx=0x7fd5000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dc94506faa68b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.131+0000 7fd4f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50002c040 con 0x5644e88ad810 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd500003d60 con 0x5644e88ad810 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e8793160 con 0x5644e88ad810 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd500020040 con 0x5644e88ad810 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 10 client.5265 fetch_fsmap learned FSMap version 16 2023-11-28T11:19:06.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5644e862a200 con 0x5644e88ad810 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5644e8793160 con 0x5644e88ad810 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd500038070 con 0x5644e88ad810 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 10 client.5265 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 10 client.5265 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 20 client.5265 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88e20f0 0x5644e88cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 4 client.5265 mds_command: new command op to 54722 tid=0[{"prefix": "status"}] 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.132+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 --> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644e877d170 con 0x5644e88e20f0 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd516d9d700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88e20f0 0x5644e88cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd516d9d700 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88e20f0 0x5644e88cb520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9367dd6cc2572b41 server_cookie=fce2028a40a57111 in_seq=0 out_seq=0 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd4f5ffb700 10 client.5265 ms_handle_connect on v2:172.21.15.53:6833/3764823575 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd4f5ffb700 1 -- 172.21.15.53:0/902366140 <== mds.0 v2:172.21.15.53:6833/3764823575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5644e877d170 con 0x5644e88e20f0 2023-11-28T11:19:06.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd4f5ffb700 10 client.5265 handle_command_reply: tid=0 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.133+0000 7fd530bb3b80 1 client.5265 shutdown 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 msgr2=0x5644e892e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd500009f20 tx=0x7fd5000047c0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 shutdown_connections 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:6833/3764823575,v1:172.21.15.53:6834/3764823575] conn(0x5644e88e20f0 0x5644e88cb520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5644e88ae120 0x5644e88e1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644e88ab130 0x5644e892d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd4f5ffb700 0 client.5265 ms_handle_reset on v2:172.21.15.53:6833/3764823575 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 --2- 172.21.15.53:0/902366140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644e88ad810 0x5644e892e1e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 >> 172.21.15.53:0/902366140 conn(0x5644e8880d60 msgr2=0x5644e88c1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 shutdown_connections 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 1 -- 172.21.15.53:0/902366140 wait complete. 2023-11-28T11:19:06.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.134+0000 7fd530bb3b80 20 client.5265 trim_cache size 0 max 16384 2023-11-28T11:19:06.141 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:06.141 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "38901cf1-eb03-4cf6-8c7d-44ad237eb202", 2023-11-28T11:19:06.141 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 54722, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.6412161540000003, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 16, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 36, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 36, 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 15.953817937 2023-11-28T11:19:06.142 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:06.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 msgr2=0x7fd4f806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 0x7fd4f806a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5000090b0 tx=0x7fd500008030 comp rx=0 tx=0).stop 2023-11-28T11:19:06.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 msgr2=0x7fd518143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e810 tx=0x7fd50800eb20 comp rx=0 tx=0).stop 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 shutdown_connections 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd4f8068320 0x7fd4f806a7d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd518161340 0x7fd5180b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5180d2320 0x7fd518143510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 --2- 172.21.15.53:0/477328390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51815c9b0 0x7fd5181e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 >> 172.21.15.53:0/477328390 conn(0x7fd518072600 msgr2=0x7fd5180b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 shutdown_connections 2023-11-28T11:19:06.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.135+0000 7fd51d1ff700 1 -- 172.21.15.53:0/477328390 wait complete. 2023-11-28T11:19:06.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:19:06.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.355+0000 7fda51fff700 1 Processor -- start 2023-11-28T11:19:06.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.355+0000 7fda51fff700 1 -- start start 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.356+0000 7fda51fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda51fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fda4c161340 0x7fda4c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda51fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0d1950 con 0x7fda4c0d2320 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda51fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0b5800 con 0x7fda4c15c9b0 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda51fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0b5520 con 0x7fda4c161340 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50680/0 (socket says 172.21.15.53:50680) 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1255366098 learned_addr learned my addr 172.21.15.53:0/1255366098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4affd700 1 --2- 172.21.15.53:0/1255366098 >> v2:172.21.15.157:3300/0 conn(0x7fda4c161340 0x7fda4c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1255366098 >> v2:172.21.15.157:3300/0 conn(0x7fda4c161340 msgr2=0x7fda4c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 --2- 172.21.15.53:0/1255366098 >> v2:172.21.15.157:3300/0 conn(0x7fda4c161340 0x7fda4c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1255366098 >> v1:172.21.15.157:6789/0 conn(0x7fda4c0d2320 legacy=0x7fda4c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.357+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1255366098 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0d1300 con 0x7fda4c15c9b0 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda4b7fe700 1 --2- 172.21.15.53:0/1255366098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0685c0 tx=0x7fda3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae815c76a3cb59a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda4a7fc700 1 -- 172.21.15.53:0/1255366098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011830 con 0x7fda4c15c9b0 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda4a7fc700 1 -- 172.21.15.53:0/1255366098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c011990 con 0x7fda4c15c9b0 2023-11-28T11:19:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda4a7fc700 1 -- 172.21.15.53:0/1255366098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011d20 con 0x7fda4c15c9b0 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda51fff700 1 -- 172.21.15.53:0/1255366098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c15c9b0 msgr2=0x7fda4c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.358+0000 7fda51fff700 1 --2- 172.21.15.53:0/1255366098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0685c0 tx=0x7fda3c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 -- 172.21.15.53:0/1255366098 shutdown_connections 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 --2- 172.21.15.53:0/1255366098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c15c9b0 0x7fda4c0b4bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 --2- 172.21.15.53:0/1255366098 >> v2:172.21.15.157:3300/0 conn(0x7fda4c161340 0x7fda4c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 -- 172.21.15.53:0/1255366098 >> 172.21.15.53:0/1255366098 conn(0x7fda4c072600 msgr2=0x7fda4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 -- 172.21.15.53:0/1255366098 shutdown_connections 2023-11-28T11:19:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.359+0000 7fda51fff700 1 -- 172.21.15.53:0/1255366098 wait complete. 2023-11-28T11:19:06.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.361+0000 7fda51fff700 1 Processor -- start 2023-11-28T11:19:06.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.361+0000 7fda51fff700 1 -- start start 2023-11-28T11:19:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.362+0000 7fda51fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda51fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda4c15c9b0 0x7fda4c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda51fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 0x7fda4c154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda51fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c150060 con 0x7fda4c0d2320 2023-11-28T11:19:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda51fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c155780 con 0x7fda4c15c9b0 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda51fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c155a60 con 0x7fda4c161340 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50682/0 (socket says 172.21.15.53:50682) 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1326998502 learned_addr learned my addr 172.21.15.53:0/1326998502 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4affd700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda4c15c9b0 0x7fda4c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4bfff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 0x7fda4c154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 msgr2=0x7fda4c154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 0x7fda4c154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda4c15c9b0 msgr2=0x7fda4c14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda4c15c9b0 0x7fda4c14fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4b7fe700 1 -- 172.21.15.53:0/1326998502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009550 con 0x7fda4c0d2320 2023-11-28T11:19:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.363+0000 7fda4bfff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 0x7fda4c154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda4b7fe700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda3c000c00 tx=0x7fda3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004480 con 0x7fda4c0d2320 2023-11-28T11:19:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c01f070 con 0x7fda4c0d2320 2023-11-28T11:19:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0038e0 con 0x7fda4c0d2320 2023-11-28T11:19:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c0b5130 con 0x7fda4c0d2320 2023-11-28T11:19:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.364+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c0d15e0 con 0x7fda4c0d2320 2023-11-28T11:19:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.365+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c0b52c0 con 0x7fda4c0d2320 2023-11-28T11:19:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.366+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda3c02a020 con 0x7fda4c0d2320 2023-11-28T11:19:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.367+0000 7fda33fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 0x7fda3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.367+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda3c087300 con 0x7fda4c0d2320 2023-11-28T11:19:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.368+0000 7fda4affd700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 0x7fda3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.369+0000 7fda4affd700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 0x7fda3806a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda40009be0 tx=0x7fda40009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.372+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda3c053a70 con 0x7fda4c0d2320 2023-11-28T11:19:06.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.618+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fda4c152bc0 con 0x7fda4c0d2320 2023-11-28T11:19:06.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.618+0000 7fda33fff700 1 -- 172.21.15.53:0/1326998502 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fda3c057090 con 0x7fda4c0d2320 2023-11-28T11:19:06.620 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:19:06.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 msgr2=0x7fda3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 0x7fda3806a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda40009be0 tx=0x7fda40009210 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 msgr2=0x7fda4c150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda3c000c00 tx=0x7fda3c004750 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 shutdown_connections 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fda380682f0 0x7fda3806a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4c161340 0x7fda4c154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c0d2320 0x7fda4c150290 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 --2- 172.21.15.53:0/1326998502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda4c15c9b0 0x7fda4c14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.620+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 >> 172.21.15.53:0/1326998502 conn(0x7fda4c072600 msgr2=0x7fda4c15a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.621+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 shutdown_connections 2023-11-28T11:19:06.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:06.621+0000 7fda51fff700 1 -- 172.21.15.53:0/1326998502 wait complete. 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:19:06.641 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:06.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:06.642 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.686 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42072sec preferred_lft 42072sec 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.687 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::892:d6ff:fe9c:c696/64 scope link 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:19:06.688 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::86e:80ff:fe63:4cd1/64 scope link 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run.smithi053.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:e4:5d:14:b9:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::68e4:5dff:fe14:b995/64 scope link 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:06.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:06.689 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:19:06.793 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:19:06.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:06.797 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-28T11:19:06.843 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-28T11:19:06.844 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:19:06.844 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:19:06.844 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:19:06.844 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:19:06.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:19:06.845 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:19:07.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.023 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:19:07.066 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:19:07.066 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:19:07.066 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:07.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-11-28T11:19:07.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:19:07.176 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:07.220 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:07.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:19:07.220 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:07.303 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:19:07.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:07.450 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:07.450 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:07.450 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:07.469 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:07.469 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:07.469 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:07.469 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:19:07.470 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:19:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:19:07.496 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:19:07.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:07.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.568 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:19:07.665 INFO:teuthology.orchestra.run.smithi053.stdout:client5274 2023-11-28T11:19:07.665 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:07.665 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:07.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.709 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:19:07.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.799 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:19:07.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.846 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:19:07.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.892 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:19:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.941 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:19:07.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:07.986 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:19:08.075 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2023-11-28T11:19:08.076 INFO:teuthology.orchestra.run.smithi053.stdout:./rfile 2023-11-28T11:19:08.076 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir 2023-11-28T11:19:08.076 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/pfile 2023-11-28T11:19:08.076 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/childdir 2023-11-28T11:19:08.076 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/childdir/cfile 2023-11-28T11:19:08.077 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:08.077 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:08.078 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:08.078 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:08.078 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:19:08.078 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:08.191 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:19:08.377 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:08.377 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:08.377 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:08.377 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:08.377 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:19:08.378 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:08.531 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2023-11-28T11:19:08.715 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:08.715 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:08.715 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:08.715 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:08.716 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:19:08.716 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:08.870 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2023-11-28T11:19:09.055 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:09.055 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:09.055 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:09.056 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:09.056 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:19:09.056 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:09.209 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628279 2023-11-28T11:19:09.393 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:09.393 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:09.393 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:09.394 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:09.394 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:19:09.394 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:09.547 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:19:09.732 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:09.885 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628280 2023-11-28T11:19:10.070 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:10.070 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:10.154 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:10.154 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:10.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:10.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.476+0000 7f5b9a9ff700 1 Processor -- start 2023-11-28T11:19:10.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.476+0000 7f5b9a9ff700 1 -- start start 2023-11-28T11:19:10.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.477+0000 7f5b9a9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b9415c9b0 0x7f5b940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b9a9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940d1950 con 0x7f5b94161340 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b9a9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940d24f0 con 0x7f5b940cd7c0 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b9a9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940b5800 con 0x7f5b9415c9b0 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b937fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5b94161340 0x7f5b940d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55158/0 (socket says 172.21.15.53:55158) 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b937fe700 1 -- 172.21.15.53:0/4055743149 learned_addr learned my addr 172.21.15.53:0/4055743149 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:10.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b93fff700 1 --2- 172.21.15.53:0/4055743149 >> v2:172.21.15.157:3300/0 conn(0x7f5b9415c9b0 0x7f5b940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b92ffd700 1 -- 172.21.15.53:0/4055743149 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151493042 0 0) 0x7f5b940d1950 con 0x7f5b94161340 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.478+0000 7f5b92ffd700 1 -- 172.21.15.53:0/4055743149 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c0035b0 con 0x7f5b94161340 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b93fff700 1 -- 172.21.15.53:0/4055743149 >> v1:172.21.15.157:6789/0 conn(0x7f5b940cd7c0 legacy=0x7f5b940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b93fff700 1 -- 172.21.15.53:0/4055743149 >> v1:172.21.15.53:6789/0 conn(0x7f5b94161340 legacy=0x7f5b940d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b93fff700 1 -- 172.21.15.53:0/4055743149 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940b5520 con 0x7f5b9415c9b0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b93fff700 1 --2- 172.21.15.53:0/4055743149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9415c9b0 0x7f5b940b4bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009840 tx=0x7f5b84009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7912187d41f12cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b92ffd700 1 -- 172.21.15.53:0/4055743149 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84011830 con 0x7f5b9415c9b0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b92ffd700 1 -- 172.21.15.53:0/4055743149 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84011990 con 0x7f5b9415c9b0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.479+0000 7f5b92ffd700 1 -- 172.21.15.53:0/4055743149 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84011d20 con 0x7f5b9415c9b0 2023-11-28T11:19:10.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/4055743149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9415c9b0 msgr2=0x7f5b940b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/4055743149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9415c9b0 0x7f5b940b4bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009840 tx=0x7f5b84009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/4055743149 shutdown_connections 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/4055743149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9415c9b0 0x7f5b940b4bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/4055743149 >> 172.21.15.53:0/4055743149 conn(0x7f5b94072600 msgr2=0x7f5b940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/4055743149 shutdown_connections 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.480+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/4055743149 wait complete. 2023-11-28T11:19:10.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.482+0000 7f5b9a9ff700 1 Processor -- start 2023-11-28T11:19:10.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.482+0000 7f5b9a9ff700 1 -- start start 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b940cd7c0 0x7f5b9414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 0x7f5b941e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9414e8f0 con 0x7f5b9415c9b0 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c9020 con 0x7f5b94161340 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b9a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c6e20 con 0x7f5b940cd7c0 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50694/0 (socket says 172.21.15.53:50694) 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 -- 172.21.15.53:0/3042321071 learned_addr learned my addr 172.21.15.53:0/3042321071 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b93fff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b940cd7c0 0x7f5b9414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b98f9c700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 0x7f5b941e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 -- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b940cd7c0 msgr2=0x7f5b9414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b940cd7c0 0x7f5b9414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 -- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 msgr2=0x7f5b941e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 0x7f5b941e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b937fe700 1 -- 172.21.15.53:0/3042321071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840094a0 con 0x7f5b9415c9b0 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.483+0000 7f5b98f9c700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 0x7f5b941e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b937fe700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c700 tx=0x7f5b8800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88005d70 con 0x7f5b9415c9b0 2023-11-28T11:19:10.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b880070d0 con 0x7f5b9415c9b0 2023-11-28T11:19:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b940d15e0 con 0x7f5b9415c9b0 2023-11-28T11:19:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007460 con 0x7f5b9415c9b0 2023-11-28T11:19:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.484+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b940d1300 con 0x7f5b9415c9b0 2023-11-28T11:19:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.485+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b940b5130 con 0x7f5b9415c9b0 2023-11-28T11:19:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.486+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b88019030 con 0x7f5b9415c9b0 2023-11-28T11:19:10.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.487+0000 7f5b90ff9700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 0x7f5b8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.487+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b88085240 con 0x7f5b9415c9b0 2023-11-28T11:19:10.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.487+0000 7f5b93fff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 0x7f5b8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.488+0000 7f5b93fff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 0x7f5b8006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b840090b0 tx=0x7f5b8401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.491+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b88051930 con 0x7f5b9415c9b0 2023-11-28T11:19:10.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.692+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b94066320 con 0x7f5b9415c9b0 2023-11-28T11:19:10.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.693+0000 7f5b90ff9700 1 -- 172.21.15.53:0/3042321071 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f5b88054f50 con 0x7f5b9415c9b0 2023-11-28T11:19:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:10.695 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:00.492727+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.695+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 msgr2=0x7f5b8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.695+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 0x7f5b8006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b840090b0 tx=0x7f5b8401c040 comp rx=0 tx=0).stop 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.695+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 msgr2=0x7f5b941e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.695+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c700 tx=0x7f5b8800cac0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 shutdown_connections 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5b800682f0 0x7f5b8006a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b940cd7c0 0x7f5b9414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b94161340 0x7f5b941e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 --2- 172.21.15.53:0/3042321071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b9415c9b0 0x7f5b941e8330 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 >> 172.21.15.53:0/3042321071 conn(0x7f5b94072600 msgr2=0x7f5b940d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 shutdown_connections 2023-11-28T11:19:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.696+0000 7f5b9a9ff700 1 -- 172.21.15.53:0/3042321071 wait complete. 2023-11-28T11:19:10.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:18:57.143723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:18:58.463907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:19:10.716 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:19:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.921+0000 7f7b2edff700 1 Processor -- start 2023-11-28T11:19:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.921+0000 7f7b2edff700 1 -- start start 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2edff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 0x7f7b280c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 0x7f7b280c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2edff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b280ccc00 con 0x7f7b28159490 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2edff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280ccec0 con 0x7f7b28159860 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280cd1a0 con 0x7f7b280b0d80 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2cb9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7b28159490 0x7f7b280b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52424/0 (socket says 172.21.15.53:52424) 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2cb9b700 1 -- 172.21.15.53:0/142307529 learned_addr learned my addr 172.21.15.53:0/142307529 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b27fff700 1 --2- 172.21.15.53:0/142307529 >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 0x7f7b280c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b2d39c700 1 --2- 172.21.15.53:0/142307529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 0x7f7b280c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b277fe700 1 -- 172.21.15.53:0/142307529 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127076670 0 0) 0x7f7b280ccc00 con 0x7f7b28159490 2023-11-28T11:19:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.923+0000 7f7b277fe700 1 -- 172.21.15.53:0/142307529 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b180035b0 con 0x7f7b28159490 2023-11-28T11:19:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b2d39c700 1 -- 172.21.15.53:0/142307529 >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 msgr2=0x7f7b280c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b2d39c700 1 --2- 172.21.15.53:0/142307529 >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 0x7f7b280c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b2d39c700 1 -- 172.21.15.53:0/142307529 >> v1:172.21.15.53:6789/0 conn(0x7f7b28159490 legacy=0x7f7b280b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b2d39c700 1 -- 172.21.15.53:0/142307529 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280cd480 con 0x7f7b280b0d80 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b27fff700 1 --2- 172.21.15.53:0/142307529 >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 0x7f7b280c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b2d39c700 1 --2- 172.21.15.53:0/142307529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 0x7f7b280c9a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009d90 tx=0x7f7b2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bdc9e34b2bb9b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b277fe700 1 -- 172.21.15.53:0/142307529 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e070 con 0x7f7b280b0d80 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b277fe700 1 -- 172.21.15.53:0/142307529 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2000bb20 con 0x7f7b280b0d80 2023-11-28T11:19:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.924+0000 7f7b277fe700 1 -- 172.21.15.53:0/142307529 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20012430 con 0x7f7b280b0d80 2023-11-28T11:19:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 -- 172.21.15.53:0/142307529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 msgr2=0x7f7b280c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 --2- 172.21.15.53:0/142307529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 0x7f7b280c9a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009d90 tx=0x7f7b2000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 -- 172.21.15.53:0/142307529 shutdown_connections 2023-11-28T11:19:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 --2- 172.21.15.53:0/142307529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b280b0d80 0x7f7b280c9a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 --2- 172.21.15.53:0/142307529 >> v2:172.21.15.157:3300/0 conn(0x7f7b28159860 0x7f7b280c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 -- 172.21.15.53:0/142307529 >> 172.21.15.53:0/142307529 conn(0x7f7b280cb670 msgr2=0x7f7b280cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 -- 172.21.15.53:0/142307529 shutdown_connections 2023-11-28T11:19:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.925+0000 7f7b2edff700 1 -- 172.21.15.53:0/142307529 wait complete. 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.927+0000 7f7b2edff700 1 Processor -- start 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.927+0000 7f7b2edff700 1 -- start start 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28159490 0x7f7b28158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b28158540 0x7f7b28155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c4a70 con 0x7f7b280b0d80 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b3970 con 0x7f7b28159490 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b3c50 con 0x7f7b28158540 2023-11-28T11:19:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43660/0 (socket says 172.21.15.53:43660) 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 -- 172.21.15.53:0/1551233801 learned_addr learned my addr 172.21.15.53:0/1551233801 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2d39c700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b28158540 0x7f7b28155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b27fff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28159490 0x7f7b28158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 -- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b28158540 msgr2=0x7f7b28155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b28158540 0x7f7b28155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 -- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28159490 msgr2=0x7f7b28158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28159490 0x7f7b28158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.928+0000 7f7b2cb9b700 1 -- 172.21.15.53:0/1551233801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b140094a0 con 0x7f7b280b0d80 2023-11-28T11:19:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b2cb9b700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c700 tx=0x7f7b1c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c005d70 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0070d0 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b20009a70 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c007460 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.929+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b280c9f40 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.930+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b280cd760 con 0x7f7b280b0d80 2023-11-28T11:19:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.931+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b1c019030 con 0x7f7b280b0d80 2023-11-28T11:19:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.932+0000 7f7b257fa700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 0x7f7b0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.932+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b1c0852f0 con 0x7f7b280b0d80 2023-11-28T11:19:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.933+0000 7f7b27fff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 0x7f7b0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.934+0000 7f7b27fff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 0x7f7b0c06a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b2806dbc0 tx=0x7f7b14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:10.936+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b1c0519e0 con 0x7f7b280b0d80 2023-11-28T11:19:11.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.139+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b280cdc80 con 0x7f7b280b0d80 2023-11-28T11:19:11.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.140+0000 7f7b257fa700 1 -- 172.21.15.53:0/1551233801 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b1c055000 con 0x7f7b280b0d80 2023-11-28T11:19:11.142 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:19:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 msgr2=0x7f7b0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 0x7f7b0c06a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b2806dbc0 tx=0x7f7b14008040 comp rx=0 tx=0).stop 2023-11-28T11:19:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 msgr2=0x7f7b280c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c700 tx=0x7f7b1c00cac0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 shutdown_connections 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7b0c0682f0 0x7f7b0c06a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b28158540 0x7f7b28155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b280b0d80 0x7f7b280c4530 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 --2- 172.21.15.53:0/1551233801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28159490 0x7f7b28158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 >> 172.21.15.53:0/1551233801 conn(0x7f7b280cb670 msgr2=0x7f7b280ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 shutdown_connections 2023-11-28T11:19:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.142+0000 7f7b2edff700 1 -- 172.21.15.53:0/1551233801 wait complete. 2023-11-28T11:19:11.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:19:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 Processor -- start 2023-11-28T11:19:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 -- start start 2023-11-28T11:19:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cf4161340 0x7f1cf40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1cf40d2320 0x7f1cf40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40b5800 con 0x7f1cf40d2320 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40d1950 con 0x7f1cf4161340 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf9bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40b5520 con 0x7f1cf415c9b0 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.367+0000 7f1cf3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1cf40d2320 0x7f1cf40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf2ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cf4161340 0x7f1cf40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1cf40d2320 0x7f1cf40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43672/0 (socket says 172.21.15.53:43672) 2023-11-28T11:19:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 -- 172.21.15.53:0/4242643478 learned_addr learned my addr 172.21.15.53:0/4242643478 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 -- 172.21.15.53:0/4242643478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 msgr2=0x7f1cf40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 --2- 172.21.15.53:0/4242643478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 -- 172.21.15.53:0/4242643478 >> v2:172.21.15.157:3300/0 conn(0x7f1cf4161340 msgr2=0x7f1cf40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 --2- 172.21.15.53:0/4242643478 >> v2:172.21.15.157:3300/0 conn(0x7f1cf4161340 0x7f1cf40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 -- 172.21.15.53:0/4242643478 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40d1300 con 0x7f1cf40d2320 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.368+0000 7f1cf3fff700 1 --2- 172.21.15.53:0/4242643478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf40cd7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1cf406bc00 tx=0x7f1cec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d97f82931ec0d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf27fc700 1 -- 172.21.15.53:0/4242643478 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00e070 con 0x7f1cf40d2320 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf27fc700 1 -- 172.21.15.53:0/4242643478 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00b910 con 0x7f1cf40d2320 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf27fc700 1 -- 172.21.15.53:0/4242643478 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00bcc0 con 0x7f1cf40d2320 2023-11-28T11:19:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 -- 172.21.15.53:0/4242643478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 msgr2=0x7f1cf40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/4242643478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf40cd7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1cf406bc00 tx=0x7f1cec00b330 comp rx=0 tx=0).stop 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 -- 172.21.15.53:0/4242643478 shutdown_connections 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/4242643478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/4242643478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf40cd7c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/4242643478 >> v2:172.21.15.157:3300/0 conn(0x7f1cf4161340 0x7f1cf40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 -- 172.21.15.53:0/4242643478 >> 172.21.15.53:0/4242643478 conn(0x7f1cf4072600 msgr2=0x7f1cf40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 -- 172.21.15.53:0/4242643478 shutdown_connections 2023-11-28T11:19:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.369+0000 7f1cf9bff700 1 -- 172.21.15.53:0/4242643478 wait complete. 2023-11-28T11:19:11.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.371+0000 7f1cf9bff700 1 Processor -- start 2023-11-28T11:19:11.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.371+0000 7f1cf9bff700 1 -- start start 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf4161340 0x7f1cf4150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41e8680 con 0x7f1cf40d2320 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41535c0 con 0x7f1cf4161340 2023-11-28T11:19:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf9bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41e8aa0 con 0x7f1cf415c9b0 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43686/0 (socket says 172.21.15.53:43686) 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 -- 172.21.15.53:0/1854586451 learned_addr learned my addr 172.21.15.53:0/1854586451 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 -- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 msgr2=0x7f1cf41502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 -- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf4161340 msgr2=0x7f1cf4150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf4161340 0x7f1cf4150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.373+0000 7f1cf37fe700 1 -- 172.21.15.53:0/1854586451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009b30 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cf37fe700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e8a0 tx=0x7f1ce400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4009590 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce400eea0 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf40b5130 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400c9f0 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.374+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf40d15e0 con 0x7f1cf40d2320 2023-11-28T11:19:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.375+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf40b52c0 con 0x7f1cf40d2320 2023-11-28T11:19:11.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.376+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ce4019030 con 0x7f1cf40d2320 2023-11-28T11:19:11.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.377+0000 7f1cdbfff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 0x7f1ce006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.377+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ce4085ba0 con 0x7f1cf40d2320 2023-11-28T11:19:11.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.377+0000 7f1cf2ffd700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 0x7f1ce006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.378+0000 7f1cf2ffd700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 0x7f1ce006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cf415f450 tx=0x7f1ce80095a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.382+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ce4052310 con 0x7f1cf40d2320 2023-11-28T11:19:11.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.584+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1cf40745c0 con 0x7f1cf40d2320 2023-11-28T11:19:11.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.584+0000 7f1cdbfff700 1 -- 172.21.15.53:0/1854586451 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ce4017020 con 0x7f1cf40d2320 2023-11-28T11:19:11.586 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:19:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 msgr2=0x7f1ce006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 0x7f1ce006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cf415f450 tx=0x7f1ce80095a0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 msgr2=0x7f1cf415f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e8a0 tx=0x7f1ce400ec60 comp rx=0 tx=0).stop 2023-11-28T11:19:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 shutdown_connections 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1ce00682f0 0x7f1ce006a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf4161340 0x7f1cf4150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf40d2320 0x7f1cf415f950 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 --2- 172.21.15.53:0/1854586451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf415c9b0 0x7f1cf41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 >> 172.21.15.53:0/1854586451 conn(0x7f1cf4072600 msgr2=0x7f1cf415ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 shutdown_connections 2023-11-28T11:19:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.587+0000 7f1cf9bff700 1 -- 172.21.15.53:0/1854586451 wait complete. 2023-11-28T11:19:11.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:19:11.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.810+0000 7f9b11dff700 1 Processor -- start 2023-11-28T11:19:11.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.810+0000 7f9b11dff700 1 -- start start 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b11dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b11dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9b0c161340 0x7f9b0c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b11dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c0d1950 con 0x7f9b0c0d2320 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b11dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b5800 con 0x7f9b0c161340 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b11dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b5520 con 0x7f9b0c15c9b0 2023-11-28T11:19:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b0affd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9b0c161340 0x7f9b0c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b0affd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9b0c161340 0x7f9b0c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43694/0 (socket says 172.21.15.53:43694) 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.811+0000 7f9b0affd700 1 -- 172.21.15.53:0/2514200185 learned_addr learned my addr 172.21.15.53:0/2514200185 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0affd700 1 -- 172.21.15.53:0/2514200185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 msgr2=0x7f9b0c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0affd700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0affd700 1 -- 172.21.15.53:0/2514200185 >> v1:172.21.15.157:6789/0 conn(0x7f9b0c0d2320 legacy=0x7f9b0c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0affd700 1 -- 172.21.15.53:0/2514200185 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0d1300 con 0x7f9b0c161340 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0affd700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c0d1de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9b000098f0 tx=0x7f9b00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f52ae09dcf6d544 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0a7fc700 1 -- 172.21.15.53:0/2514200185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000b8a0 con 0x7f9b0c161340 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0a7fc700 1 -- 172.21.15.53:0/2514200185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000ba00 con 0x7f9b0c161340 2023-11-28T11:19:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.812+0000 7f9b0a7fc700 1 -- 172.21.15.53:0/2514200185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000bd90 con 0x7f9b0c161340 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 -- 172.21.15.53:0/2514200185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 msgr2=0x7f9b0c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c0d1de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9b000098f0 tx=0x7f9b00009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 -- 172.21.15.53:0/2514200185 shutdown_connections 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 --2- 172.21.15.53:0/2514200185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c0d1de0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 -- 172.21.15.53:0/2514200185 >> 172.21.15.53:0/2514200185 conn(0x7f9b0c072600 msgr2=0x7f9b0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 -- 172.21.15.53:0/2514200185 shutdown_connections 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.813+0000 7f9b11dff700 1 -- 172.21.15.53:0/2514200185 wait complete. 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.815+0000 7f9b11dff700 1 Processor -- start 2023-11-28T11:19:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.815+0000 7f9b11dff700 1 -- start start 2023-11-28T11:19:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 0x7f9b0c1e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c1e7c30 con 0x7f9b0c161340 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c1592a0 con 0x7f9b0c0d2320 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b11dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c159580 con 0x7f9b0c15c9b0 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43698/0 (socket says 172.21.15.53:43698) 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.816+0000 7f9b0bfff700 1 -- 172.21.15.53:0/1203664351 learned_addr learned my addr 172.21.15.53:0/1203664351 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0affd700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 0x7f9b0c1e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 -- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 msgr2=0x7f9b0c156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 -- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 msgr2=0x7f9b0c1e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 0x7f9b0c1e73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 -- 172.21.15.53:0/1203664351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b00009550 con 0x7f9b0c161340 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 0x7f9b0c1e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b0bfff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400cb50 tx=0x7f9b0400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04003c00 con 0x7f9b0c161340 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04003d60 con 0x7f9b0c161340 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c0b5130 con 0x7f9b0c161340 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400ab70 con 0x7f9b0c161340 2023-11-28T11:19:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.817+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c0d15e0 con 0x7f9b0c161340 2023-11-28T11:19:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.818+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c0b52c0 con 0x7f9b0c161340 2023-11-28T11:19:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.819+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b04010050 con 0x7f9b0c161340 2023-11-28T11:19:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.820+0000 7f9af3fff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 0x7f9af806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.820+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b04085c60 con 0x7f9b0c161340 2023-11-28T11:19:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.821+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 0x7f9af806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.821+0000 7f9b0b7fe700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 0x7f9af806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009a60 tx=0x7f9afc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:11.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:11.825+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b040523d0 con 0x7f9b0c161340 2023-11-28T11:19:12.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.027+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b0c159a70 con 0x7f9b0c161340 2023-11-28T11:19:12.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.027+0000 7f9af3fff700 1 -- 172.21.15.53:0/1203664351 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b040559f0 con 0x7f9b0c161340 2023-11-28T11:19:12.028 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:19:12.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 msgr2=0x7f9af806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 0x7f9af806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009a60 tx=0x7f9afc008040 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 msgr2=0x7f9b0c157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400cb50 tx=0x7f9b0400cf10 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 shutdown_connections 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9af80682f0 0x7f9af806a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c161340 0x7f9b0c157410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0c0d2320 0x7f9b0c1e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 --2- 172.21.15.53:0/1203664351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b0c15c9b0 0x7f9b0c156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 >> 172.21.15.53:0/1203664351 conn(0x7f9b0c072600 msgr2=0x7f9b0c15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.029+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 shutdown_connections 2023-11-28T11:19:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.030+0000 7f9b11dff700 1 -- 172.21.15.53:0/1203664351 wait complete. 2023-11-28T11:19:12.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:19:12.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.253+0000 7fd7275ff700 1 Processor -- start 2023-11-28T11:19:12.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.254+0000 7fd7275ff700 1 -- start start 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd7275ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 0x7fd7200cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd7275ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200d1950 con 0x7fd72015c9b0 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd7275ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200ce140 con 0x7fd720161340 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd7275ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200b5800 con 0x7fd7200cd7c0 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd72539b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd72015c9b0 0x7fd7200b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52432/0 (socket says 172.21.15.53:52432) 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd72539b700 1 -- 172.21.15.53:0/2361169348 learned_addr learned my addr 172.21.15.53:0/2361169348 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:12.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.255+0000 7fd725b9c700 1 --2- 172.21.15.53:0/2361169348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 0x7fd7200cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867948170 0 0) 0x7fd7200d1950 con 0x7fd72015c9b0 2023-11-28T11:19:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c0035b0 con 0x7fd72015c9b0 2023-11-28T11:19:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245501631 0 0) 0x7fd7200ce140 con 0x7fd720161340 2023-11-28T11:19:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7200d1950 con 0x7fd720161340 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081329955 0 0) 0x7fd70c0035b0 con 0x7fd72015c9b0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd725b9c700 1 -- 172.21.15.53:0/2361169348 >> v1:172.21.15.157:6789/0 conn(0x7fd720161340 legacy=0x7fd7200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd725b9c700 1 -- 172.21.15.53:0/2361169348 >> v1:172.21.15.53:6789/0 conn(0x7fd72015c9b0 legacy=0x7fd7200b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd725b9c700 1 -- 172.21.15.53:0/2361169348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200d24f0 con 0x7fd7200cd7c0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.256+0000 7fd725b9c700 1 --2- 172.21.15.53:0/2361169348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 0x7fd7200cdc00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd718009dc0 tx=0x7fd71800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a698e4a865493f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800e070 con 0x7fd7200cd7c0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718003a20 con 0x7fd7200cd7c0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd717fff700 1 -- 172.21.15.53:0/2361169348 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c001670 con 0x7fd7200cd7c0 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 -- 172.21.15.53:0/2361169348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 msgr2=0x7fd7200cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 --2- 172.21.15.53:0/2361169348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 0x7fd7200cdc00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd718009dc0 tx=0x7fd71800b330 comp rx=0 tx=0).stop 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 -- 172.21.15.53:0/2361169348 shutdown_connections 2023-11-28T11:19:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 --2- 172.21.15.53:0/2361169348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7200cd7c0 0x7fd7200cdc00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 -- 172.21.15.53:0/2361169348 >> 172.21.15.53:0/2361169348 conn(0x7fd720072600 msgr2=0x7fd7200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 -- 172.21.15.53:0/2361169348 shutdown_connections 2023-11-28T11:19:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.257+0000 7fd7275ff700 1 -- 172.21.15.53:0/2361169348 wait complete. 2023-11-28T11:19:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.259+0000 7fd7275ff700 1 Processor -- start 2023-11-28T11:19:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.259+0000 7fd7275ff700 1 -- start start 2023-11-28T11:19:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7200cd7c0 0x7fd7200cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd72015c9b0 0x7fd7200cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200cc980 con 0x7fd720161340 2023-11-28T11:19:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200cb2b0 con 0x7fd7200cd7c0 2023-11-28T11:19:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd7275ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200cbd00 con 0x7fd72015c9b0 2023-11-28T11:19:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.260+0000 7fd725b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43704/0 (socket says 172.21.15.53:43704) 2023-11-28T11:19:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 -- 172.21.15.53:0/947826037 learned_addr learned my addr 172.21.15.53:0/947826037 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd72539b700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7200cd7c0 0x7fd7200cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd724b9a700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd72015c9b0 0x7fd7200cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 -- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd72015c9b0 msgr2=0x7fd7200cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd72015c9b0 0x7fd7200cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 -- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7200cd7c0 msgr2=0x7fd7200cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7200cd7c0 0x7fd7200cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 -- 172.21.15.53:0/947826037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718009a70 con 0x7fd720161340 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd725b9c700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd718009d90 tx=0x7fd718004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.261+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180043c0 con 0x7fd720161340 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.262+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718004540 con 0x7fd720161340 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.262+0000 7fd7275ff700 1 -- 172.21.15.53:0/947826037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7200d1300 con 0x7fd720161340 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.262+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180048f0 con 0x7fd720161340 2023-11-28T11:19:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.262+0000 7fd7275ff700 1 -- 172.21.15.53:0/947826037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7200b5520 con 0x7fd720161340 2023-11-28T11:19:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.263+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7200d15e0 con 0x7fd720161340 2023-11-28T11:19:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.264+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd718018040 con 0x7fd720161340 2023-11-28T11:19:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.264+0000 7fd715ffb700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 0x7fd70406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.264+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd718087370 con 0x7fd720161340 2023-11-28T11:19:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.265+0000 7fd72539b700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 0x7fd70406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.266+0000 7fd72539b700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 0x7fd70406a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd71c000f60 tx=0x7fd71c001490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.269+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd718053a80 con 0x7fd720161340 2023-11-28T11:19:12.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.472+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7200ccac0 con 0x7fd720161340 2023-11-28T11:19:12.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.473+0000 7fd715ffb700 1 -- 172.21.15.53:0/947826037 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7200ccac0 con 0x7fd720161340 2023-11-28T11:19:12.474 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:19:12.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 msgr2=0x7fd70406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 0x7fd70406a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd71c000f60 tx=0x7fd71c001490 comp rx=0 tx=0).stop 2023-11-28T11:19:12.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 msgr2=0x7fd720156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd718009d90 tx=0x7fd718004ad0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 shutdown_connections 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7040682f0 0x7fd70406a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd720161340 0x7fd720156eb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7200cd7c0 0x7fd7200cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 --2- 172.21.15.53:0/947826037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd72015c9b0 0x7fd7200cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 >> 172.21.15.53:0/947826037 conn(0x7fd720072600 msgr2=0x7fd72006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 shutdown_connections 2023-11-28T11:19:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.475+0000 7fd70b7fe700 1 -- 172.21.15.53:0/947826037 wait complete. 2023-11-28T11:19:12.494 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:19:12.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.696+0000 7f67741ff700 1 Processor -- start 2023-11-28T11:19:12.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.697+0000 7f67741ff700 1 -- start start 2023-11-28T11:19:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f67741ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c15c9b0 0x7f676c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f67741ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f676c0cd7c0 0x7f676c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f67741ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0ce160 con 0x7f676c161340 2023-11-28T11:19:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f67741ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c0b5800 con 0x7f676c0cd7c0 2023-11-28T11:19:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f67741ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0d1950 con 0x7f676c15c9b0 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f6771f9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c15c9b0 0x7f676c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f677279c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f676c0cd7c0 0x7f676c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f677279c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f676c0cd7c0 0x7f676c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:57518/0 (socket says 172.21.15.53:57518) 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.698+0000 7f677279c700 1 -- 172.21.15.53:0/1794639057 learned_addr learned my addr 172.21.15.53:0/1794639057 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f677279c700 1 -- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c15c9b0 msgr2=0x7f676c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f677279c700 1 --2- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c15c9b0 0x7f676c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f677279c700 1 -- 172.21.15.53:0/1794639057 >> v1:172.21.15.157:6789/0 conn(0x7f676c161340 legacy=0x7f676c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f677279c700 1 -- 172.21.15.53:0/1794639057 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0d24f0 con 0x7f676c0cd7c0 2023-11-28T11:19:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f677279c700 1 --2- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c0cd7c0 0x7f676c0cdc20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e50 tx=0x7f676400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb814b735cfeb769 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f6770f99700 1 -- 172.21.15.53:0/1794639057 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400e070 con 0x7f676c0cd7c0 2023-11-28T11:19:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f6770f99700 1 -- 172.21.15.53:0/1794639057 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400b910 con 0x7f676c0cd7c0 2023-11-28T11:19:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.699+0000 7f6770f99700 1 -- 172.21.15.53:0/1794639057 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400bcc0 con 0x7f676c0cd7c0 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 -- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c0cd7c0 msgr2=0x7f676c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 --2- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c0cd7c0 0x7f676c0cdc20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e50 tx=0x7f676400b330 comp rx=0 tx=0).stop 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 -- 172.21.15.53:0/1794639057 shutdown_connections 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 --2- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c0cd7c0 0x7f676c0cdc20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 --2- 172.21.15.53:0/1794639057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c15c9b0 0x7f676c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 -- 172.21.15.53:0/1794639057 >> 172.21.15.53:0/1794639057 conn(0x7f676c072600 msgr2=0x7f676c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 -- 172.21.15.53:0/1794639057 shutdown_connections 2023-11-28T11:19:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.700+0000 7f67741ff700 1 -- 172.21.15.53:0/1794639057 wait complete. 2023-11-28T11:19:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.702+0000 7f67741ff700 1 Processor -- start 2023-11-28T11:19:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.702+0000 7f67741ff700 1 -- start start 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 0x7f676c154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c161340 0x7f676c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c151b30 con 0x7f676c15c9b0 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1569e0 con 0x7f676c161340 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f67741ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c156cc0 con 0x7f676c0cd7c0 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f677179a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f677179a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43712/0 (socket says 172.21.15.53:43712) 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f677179a700 1 -- 172.21.15.53:0/2384241556 learned_addr learned my addr 172.21.15.53:0/2384241556 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f6771f9b700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 0x7f676c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.703+0000 7f677279c700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c161340 0x7f676c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 -- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c161340 msgr2=0x7f676c15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c161340 0x7f676c15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 -- 172.21.15.53:0/2384241556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 msgr2=0x7f676c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 -- 172.21.15.53:0/2384241556 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009b30 con 0x7f676c0cd7c0 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f677179a700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f6771f9b700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 0x7f676c154820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f676800e870 tx=0x7f676800eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768013680 con 0x7f676c0cd7c0 2023-11-28T11:19:12.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67680137e0 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768013b90 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.704+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c0d1300 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.705+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c0b5520 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.705+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c0d15e0 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.707+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6768019030 con 0x7f676c0cd7c0 2023-11-28T11:19:12.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.708+0000 7f67627fc700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 0x7f675006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:12.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.708+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6768086230 con 0x7f676c0cd7c0 2023-11-28T11:19:12.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.708+0000 7f677179a700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 0x7f675006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:12.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.709+0000 7f677179a700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 0x7f675006a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f676c1e9240 tx=0x7f675c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.712+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6768052990 con 0x7f676c0cd7c0 2023-11-28T11:19:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.928+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f676c156e00 con 0x7f676c0cd7c0 2023-11-28T11:19:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.929+0000 7f67627fc700 1 -- 172.21.15.53:0/2384241556 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f676c156e00 con 0x7f676c0cd7c0 2023-11-28T11:19:12.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 msgr2=0x7f675006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 0x7f675006a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f676c1e9240 tx=0x7f675c009040 comp rx=0 tx=0).stop 2023-11-28T11:19:12.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 msgr2=0x7f676c154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:12.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 0x7f676c154820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f676800e870 tx=0x7f676800eb80 comp rx=0 tx=0).stop 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 shutdown_connections 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67500682f0 0x7f675006a7a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f676c0cd7c0 0x7f676c154820 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f676c161340 0x7f676c15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 --2- 172.21.15.53:0/2384241556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c15c9b0 0x7f676c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 >> 172.21.15.53:0/2384241556 conn(0x7f676c072600 msgr2=0x7f676c0b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 shutdown_connections 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:12.931+0000 7f67741ff700 1 -- 172.21.15.53:0/2384241556 wait complete. 2023-11-28T11:19:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:19:12.950 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:19:12.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:12.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:13.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:19:13.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:13.060 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:19:13.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:13.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.312+0000 7f94b3bff700 1 Processor -- start 2023-11-28T11:19:13.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.312+0000 7f94b3bff700 1 -- start start 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.313+0000 7f94b3bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.313+0000 7f94b3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac161340 0x7f94ac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.313+0000 7f94b3bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac0d1950 con 0x7f94ac0d2320 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.313+0000 7f94b3bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0b5800 con 0x7f94ac15c9b0 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.313+0000 7f94b3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b5520 con 0x7f94ac161340 2023-11-28T11:19:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43726/0 (socket says 172.21.15.53:43726) 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 -- 172.21.15.53:0/3024062676 learned_addr learned my addr 172.21.15.53:0/3024062676 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b119a700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac161340 0x7f94ac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 -- 172.21.15.53:0/3024062676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac161340 msgr2=0x7f94ac0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac161340 0x7f94ac0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 -- 172.21.15.53:0/3024062676 >> v1:172.21.15.157:6789/0 conn(0x7f94ac0d2320 legacy=0x7f94ac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 -- 172.21.15.53:0/3024062676 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0d1300 con 0x7f94ac15c9b0 2023-11-28T11:19:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.314+0000 7f94b199b700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94a80098f0 tx=0x7f94a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4508c94b74b7e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b0999700 1 -- 172.21.15.53:0/3024062676 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800b8a0 con 0x7f94ac15c9b0 2023-11-28T11:19:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b0999700 1 -- 172.21.15.53:0/3024062676 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a800ba00 con 0x7f94ac15c9b0 2023-11-28T11:19:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b0999700 1 -- 172.21.15.53:0/3024062676 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800bd90 con 0x7f94ac15c9b0 2023-11-28T11:19:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 -- 172.21.15.53:0/3024062676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 msgr2=0x7f94ac0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94a80098f0 tx=0x7f94a8009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 -- 172.21.15.53:0/3024062676 shutdown_connections 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac0b4bf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 --2- 172.21.15.53:0/3024062676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac161340 0x7f94ac0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 -- 172.21.15.53:0/3024062676 >> 172.21.15.53:0/3024062676 conn(0x7f94ac072600 msgr2=0x7f94ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 -- 172.21.15.53:0/3024062676 shutdown_connections 2023-11-28T11:19:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.315+0000 7f94b3bff700 1 -- 172.21.15.53:0/3024062676 wait complete. 2023-11-28T11:19:13.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.317+0000 7f94b3bff700 1 Processor -- start 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.317+0000 7f94b3bff700 1 -- start start 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 0x7f94ac151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 0x7f94ac14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1554a0 con 0x7f94ac15c9b0 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1529b0 con 0x7f94ac161340 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1523f0 con 0x7f94ac0d2320 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.318+0000 7f94b119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43732/0 (socket says 172.21.15.53:43732) 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 -- 172.21.15.53:0/2417516259 learned_addr learned my addr 172.21.15.53:0/2417516259 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:13.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b199b700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 0x7f94ac151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b219c700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 0x7f94ac14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 -- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 msgr2=0x7f94ac151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 0x7f94ac151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 -- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 msgr2=0x7f94ac14cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 0x7f94ac14cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 -- 172.21.15.53:0/2417516259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8009550 con 0x7f94ac15c9b0 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b199b700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 0x7f94ac151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b219c700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 0x7f94ac14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b119a700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e810 tx=0x7f949c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:13.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c013610 con 0x7f94ac15c9b0 2023-11-28T11:19:13.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c013770 con 0x7f94ac15c9b0 2023-11-28T11:19:13.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c013b20 con 0x7f94ac15c9b0 2023-11-28T11:19:13.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.319+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac0b5130 con 0x7f94ac15c9b0 2023-11-28T11:19:13.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.320+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0d15e0 con 0x7f94ac15c9b0 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.320+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0b52c0 con 0x7f94ac15c9b0 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.321+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f949c023070 con 0x7f94ac15c9b0 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.322+0000 7f94a27fc700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 0x7f949006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.322+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f949c085a60 con 0x7f94ac15c9b0 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.323+0000 7f94b199b700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 0x7f949006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:13.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.324+0000 7f94b199b700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 0x7f949006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009c40 tx=0x7f94a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:13.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.326+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f949c052150 con 0x7f94ac15c9b0 2023-11-28T11:19:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.575+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94ac0745c0 con 0x7f94ac15c9b0 2023-11-28T11:19:13.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.576+0000 7f94a27fc700 1 -- 172.21.15.53:0/2417516259 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f949c055770 con 0x7f94ac15c9b0 2023-11-28T11:19:13.577 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:13.577 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 msgr2=0x7f949006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 0x7f949006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009c40 tx=0x7f94a8008030 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 msgr2=0x7f94ac154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e810 tx=0x7f949c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 shutdown_connections 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f94900682f0 0x7f949006a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94ac0d2320 0x7f94ac151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ac161340 0x7f94ac14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 --2- 172.21.15.53:0/2417516259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac15c9b0 0x7f94ac154f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.578+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 >> 172.21.15.53:0/2417516259 conn(0x7f94ac072600 msgr2=0x7f94ac15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.579+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 shutdown_connections 2023-11-28T11:19:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:13.579+0000 7f94b3bff700 1 -- 172.21.15.53:0/2417516259 wait complete. 2023-11-28T11:19:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:13.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:19:13.831 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2023-11-28T11:19:13.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:19:13.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:14.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.033+0000 7f27643ff700 1 Processor -- start 2023-11-28T11:19:14.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.033+0000 7f27643ff700 1 -- start start 2023-11-28T11:19:14.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.034+0000 7f27643ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.034+0000 7f27643ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f27643ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c0ce160 con 0x7f275c161340 2023-11-28T11:19:14.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f27643ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c0b5800 con 0x7f275c0cd7c0 2023-11-28T11:19:14.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f27643ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0d1950 con 0x7f275c15c9b0 2023-11-28T11:19:14.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276299c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276219b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276199a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f275c161340 0x7f275c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:39338/0 (socket says 172.21.15.53:39338) 2023-11-28T11:19:14.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276219b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:44536/0 (socket says 172.21.15.53:44536) 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276299c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:57542/0 (socket says 172.21.15.53:57542) 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.035+0000 7f276199a700 1 -- 172.21.15.53:0/4152785096 learned_addr learned my addr 172.21.15.53:0/4152785096 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276219b700 1 -- 172.21.15.53:0/4152785096 >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 msgr2=0x7f275c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276219b700 1 --2- 172.21.15.53:0/4152785096 >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276219b700 1 -- 172.21.15.53:0/4152785096 >> v1:172.21.15.157:6789/0 conn(0x7f275c161340 legacy=0x7f275c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276219b700 1 -- 172.21.15.53:0/4152785096 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0d24f0 con 0x7f275c15c9b0 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276299c700 1 --2- 172.21.15.53:0/4152785096 >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f276219b700 1 --2- 172.21.15.53:0/4152785096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2758009840 tx=0x7f2758009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a061985c4dfaea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f2761199700 1 -- 172.21.15.53:0/4152785096 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758011830 con 0x7f275c15c9b0 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f2761199700 1 -- 172.21.15.53:0/4152785096 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2758011990 con 0x7f275c15c9b0 2023-11-28T11:19:14.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.036+0000 7f2761199700 1 -- 172.21.15.53:0/4152785096 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758011d20 con 0x7f275c15c9b0 2023-11-28T11:19:14.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 -- 172.21.15.53:0/4152785096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 msgr2=0x7f275c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 --2- 172.21.15.53:0/4152785096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2758009840 tx=0x7f2758009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 -- 172.21.15.53:0/4152785096 shutdown_connections 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 --2- 172.21.15.53:0/4152785096 >> v2:172.21.15.157:3300/0 conn(0x7f275c0cd7c0 0x7f275c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 --2- 172.21.15.53:0/4152785096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c0b4bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 -- 172.21.15.53:0/4152785096 >> 172.21.15.53:0/4152785096 conn(0x7f275c072600 msgr2=0x7f275c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 -- 172.21.15.53:0/4152785096 shutdown_connections 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.037+0000 7f27643ff700 1 -- 172.21.15.53:0/4152785096 wait complete. 2023-11-28T11:19:14.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.039+0000 7f27643ff700 1 Processor -- start 2023-11-28T11:19:14.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.039+0000 7f27643ff700 1 -- start start 2023-11-28T11:19:14.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c14a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 0x7f275c0c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1f2ea0 con 0x7f275c0cd7c0 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1f3180 con 0x7f275c161340 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f27643ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1f3460 con 0x7f275c15c9b0 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f276219b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f276219b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43742/0 (socket says 172.21.15.53:43742) 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.040+0000 7f276219b700 1 -- 172.21.15.53:0/2963799826 learned_addr learned my addr 172.21.15.53:0/2963799826 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276199a700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c14a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276299c700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 0x7f275c0c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 -- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 msgr2=0x7f275c14a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c14a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 -- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 msgr2=0x7f275c0c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 0x7f275c0c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 -- 172.21.15.53:0/2963799826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754009a70 con 0x7f275c0cd7c0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276299c700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 0x7f275c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f276219b700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2758009fd0 tx=0x7f2758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27580036a0 con 0x7f275c0cd7c0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275801e070 con 0x7f275c0cd7c0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27580094a0 con 0x7f275c0cd7c0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758003a60 con 0x7f275c0cd7c0 2023-11-28T11:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.041+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c0d1300 con 0x7f275c0cd7c0 2023-11-28T11:19:14.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.042+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0b5520 con 0x7f275c0cd7c0 2023-11-28T11:19:14.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.043+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f275800f050 con 0x7f275c0cd7c0 2023-11-28T11:19:14.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.044+0000 7f2752ffd700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 0x7f274006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:14.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.044+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2758086e10 con 0x7f275c0cd7c0 2023-11-28T11:19:14.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.044+0000 7f276199a700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 0x7f274006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:14.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.045+0000 7f276199a700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 0x7f274006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f275c0c6ea0 tx=0x7f274c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:14.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.048+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f275808a030 con 0x7f275c0cd7c0 2023-11-28T11:19:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.290+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f275c0d15e0 con 0x7f275c0cd7c0 2023-11-28T11:19:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.291+0000 7f2752ffd700 1 -- 172.21.15.53:0/2963799826 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f2758015090 con 0x7f275c0cd7c0 2023-11-28T11:19:14.292 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:14.292 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 msgr2=0x7f274006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 0x7f274006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f275c0c6ea0 tx=0x7f274c009040 comp rx=0 tx=0).stop 2023-11-28T11:19:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 msgr2=0x7f275c0c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2758009fd0 tx=0x7f2758004750 comp rx=0 tx=0).stop 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 shutdown_connections 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f27400682f0 0x7f274006a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f275c161340 0x7f275c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f275c0cd7c0 0x7f275c0c98d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 --2- 172.21.15.53:0/2963799826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f275c15c9b0 0x7f275c14a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 >> 172.21.15.53:0/2963799826 conn(0x7f275c072600 msgr2=0x7f275c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 shutdown_connections 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:14.293+0000 7f27643ff700 1 -- 172.21.15.53:0/2963799826 wait complete. 2023-11-28T11:19:14.295 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:14.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-28T11:19:14.539 INFO:teuthology.orchestra.run.smithi053.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:19:14.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:19:14.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:14.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:24.540 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:19:24.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.634+0000 7f9b4edc2080 1 Processor -- start 2023-11-28T11:19:24.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.634+0000 7f9b4edc2080 1 -- start start 2023-11-28T11:19:24.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x5605594c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055939a000 con 0x5605594ca250 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055932cfe0 con 0x56055943a920 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560559438ba0 con 0x5605594ca620 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b3fb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.635+0000 7f9b3fb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39474/0 (socket says 172.21.15.53:39474) 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/174989102 learned_addr learned my addr 172.21.15.53:0/174989102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b4039b700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b40b9c700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x5605594c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 msgr2=0x5605594c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x5605594c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 msgr2=0x5605594c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/174989102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3800a100 con 0x5605594ca250 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b40b9c700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x5605594c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b34001d70 tx=0x7f9b3400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc0f7fae4afdcec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.636+0000 7f9b3eb98700 1 -- 172.21.15.53:0/174989102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34014070 con 0x5605594ca250 2023-11-28T11:19:24.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b3eb98700 1 -- 172.21.15.53:0/174989102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34008030 con 0x5605594ca250 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b3eb98700 1 -- 172.21.15.53:0/174989102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400bc40 con 0x5605594ca250 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 -- 172.21.15.53:0/174989102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 msgr2=0x5605594cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b34001d70 tx=0x7f9b3400b4c0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 -- 172.21.15.53:0/174989102 shutdown_connections 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x5605594c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/174989102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594cb8b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 -- 172.21.15.53:0/174989102 >> 172.21.15.53:0/174989102 conn(0x56055943e9d0 msgr2=0x5605594cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 -- 172.21.15.53:0/174989102 shutdown_connections 2023-11-28T11:19:24.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.637+0000 7f9b4edc2080 1 -- 172.21.15.53:0/174989102 wait complete. 2023-11-28T11:19:24.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.638+0000 7f9b4edc2080 1 Processor -- start 2023-11-28T11:19:24.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.638+0000 7f9b4edc2080 1 -- start start 2023-11-28T11:19:24.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x56055941e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055941de50 con 0x5605594ca250 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055941e130 con 0x56055943a920 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.639+0000 7f9b4edc2080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x56055941e410 con 0x5605594ca620 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39480/0 (socket says 172.21.15.53:39480) 2023-11-28T11:19:24.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/4288270656 learned_addr learned my addr 172.21.15.53:0/4288270656 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b4039b700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b40b9c700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x56055941e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 msgr2=0x56055941e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5605594ca620 0x56055941e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 msgr2=0x5605594cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56055943a920 0x5605594cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 -- 172.21.15.53:0/4288270656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3400a040 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.640+0000 7f9b3fb9a700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594ce140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b34002000 tx=0x7f9b34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.641+0000 7f9b3db96700 1 -- 172.21.15.53:0/4288270656 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34014040 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.641+0000 7f9b3db96700 1 -- 172.21.15.53:0/4288270656 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3400f990 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.641+0000 7f9b3db96700 1 -- 172.21.15.53:0/4288270656 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400fd40 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.641+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560559338830 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.641+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560559345360 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.643+0000 7f9b3db96700 1 -- 172.21.15.53:0/4288270656 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b3401b430 con 0x5605594ca250 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b3db96700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9b2c066c40 0x7f9b2c0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b3db96700 1 -- 172.21.15.53:0/4288270656 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b34085ca0 con 0x5605594ca250 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x5605594ce680 0x5605594372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x560559419c00 con 0x5605594ce680 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b4039b700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9b2c066c40 0x7f9b2c0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.644+0000 7f9b40b9c700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x5605594ce680 0x5605594372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.645+0000 7f9b4039b700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9b2c066c40 0x7f9b2c0690f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b30000c00 tx=0x7f9b30001730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.645+0000 7f9b40b9c700 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x5605594ce680 0x5605594372b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b40b9c700 1 -- 172.21.15.53:0/4288270656 <== osd.7 v2:172.21.15.157:6824/132962 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f9b3800fa60 con 0x5605594ce680 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x5605594ce680 msgr2=0x5605594372b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x5605594ce680 0x5605594372b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9b2c066c40 msgr2=0x7f9b2c0690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9b2c066c40 0x7f9b2c0690f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b30000c00 tx=0x7f9b30001730 comp rx=0 tx=0).stop 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 msgr2=0x5605594ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 --2- 172.21.15.53:0/4288270656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605594ca250 0x5605594ce140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b34002000 tx=0x7f9b34000bc0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b40b9c700 1 -- 172.21.15.53:0/4288270656 reap_dead start 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 shutdown_connections 2023-11-28T11:19:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 >> 172.21.15.53:0/4288270656 conn(0x56055943e9d0 msgr2=0x5605594c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 shutdown_connections 2023-11-28T11:19:24.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.646+0000 7f9b4edc2080 1 -- 172.21.15.53:0/4288270656 wait complete. 2023-11-28T11:19:24.655 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2023-11-28T11:19:24.656 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:19:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.743+0000 7fe0484e6080 1 Processor -- start 2023-11-28T11:19:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 -- start start 2023-11-28T11:19:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 0x558cfd9e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfd8b7000 con 0x558cfd9e94a0 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfd849fe0 con 0x558cfd95a9f0 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.744+0000 7fe0484e6080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cfd9e8f40 con 0x7fe024002260 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39496/0 (socket says 172.21.15.53:39496) 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 -- 172.21.15.53:0/4148671241 learned_addr learned my addr 172.21.15.53:0/4148671241 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03939a700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe039b9b700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 0x558cfd9e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 -- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 msgr2=0x558cfd9e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 0x558cfd9e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 -- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 msgr2=0x558cfd9dd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9dd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 -- 172.21.15.53:0/4148671241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cfd862360 con 0x558cfd9e94a0 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03939a700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9dd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe039b9b700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 0x558cfd9e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe03a39c700 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558cfd957da0 tx=0x7fe034006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7424a8065495a4c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.745+0000 7fe02bfff700 1 -- 172.21.15.53:0/4148671241 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034006b60 con 0x558cfd9e94a0 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe02bfff700 1 -- 172.21.15.53:0/4148671241 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034006cc0 con 0x558cfd9e94a0 2023-11-28T11:19:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe02bfff700 1 -- 172.21.15.53:0/4148671241 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034003650 con 0x558cfd9e94a0 2023-11-28T11:19:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 -- 172.21.15.53:0/4148671241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 msgr2=0x558cfd9e9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558cfd957da0 tx=0x7fe034006060 comp rx=0 tx=0).stop 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 -- 172.21.15.53:0/4148671241 shutdown_connections 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe024002260 0x558cfd9e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558cfd9e94a0 0x558cfd9e9920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 --2- 172.21.15.53:0/4148671241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9dd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 -- 172.21.15.53:0/4148671241 >> 172.21.15.53:0/4148671241 conn(0x558cfd95b9d0 msgr2=0x558cfd95a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 -- 172.21.15.53:0/4148671241 shutdown_connections 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.746+0000 7fe0484e6080 1 -- 172.21.15.53:0/4148671241 wait complete. 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.747+0000 7fe0484e6080 1 Processor -- start 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.747+0000 7fe0484e6080 1 -- start start 2023-11-28T11:19:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558cfd9e94a0 0x558cfd9e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 0x558cfd953180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfd9dab60 con 0x7fe024002260 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfd9536c0 con 0x558cfd95a9f0 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe0484e6080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cfd9eb580 con 0x558cfd9e94a0 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe03a39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 0x558cfd953180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe03a39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 0x558cfd953180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39500/0 (socket says 172.21.15.53:39500) 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe03a39c700 1 -- 172.21.15.53:0/3157353683 learned_addr learned my addr 172.21.15.53:0/3157353683 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe039b9b700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.748+0000 7fe03939a700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558cfd9e94a0 0x558cfd9e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 -- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558cfd9e94a0 msgr2=0x558cfd9e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558cfd9e94a0 0x558cfd9e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 -- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 msgr2=0x558cfd9e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558cfd95a9f0 0x558cfd9e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 -- 172.21.15.53:0/3157353683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c005d20 con 0x7fe024002260 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe03a39c700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 0x558cfd953180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe034006460 tx=0x7fe034001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe02affd700 1 -- 172.21.15.53:0/3157353683 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034022070 con 0x7fe024002260 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558cfd957440 con 0x7fe024002260 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe02affd700 1 -- 172.21.15.53:0/3157353683 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03401d620 con 0x7fe024002260 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe02affd700 1 -- 172.21.15.53:0/3157353683 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03401d9b0 con 0x7fe024002260 2023-11-28T11:19:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.749+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cfd855830 con 0x7fe024002260 2023-11-28T11:19:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.751+0000 7fe02affd700 1 -- 172.21.15.53:0/3157353683 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe03401db10 con 0x7fe024002260 2023-11-28T11:19:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.752+0000 7fe02affd700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe01c066c10 0x7fe01c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.752+0000 7fe02affd700 1 -- 172.21.15.53:0/3157353683 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe034085b70 con 0x7fe024002260 2023-11-28T11:19:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.752+0000 7fe0484e6080 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x558cfd93c870 0x558cfd93ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.752+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x558cfd9d99d0 con 0x558cfd93c870 2023-11-28T11:19:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.753+0000 7fe039b9b700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe01c066c10 0x7fe01c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.753+0000 7fe03939a700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x558cfd93c870 0x558cfd93ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe039b9b700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe01c066c10 0x7fe01c0690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0099b0 tx=0x7fe02c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe03939a700 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x558cfd93c870 0x558cfd93ccb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe03939a700 1 -- 172.21.15.53:0/3157353683 <== osd.5 v2:172.21.15.157:6808/132918 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fe030001ac0 con 0x558cfd93c870 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x558cfd93c870 msgr2=0x558cfd93ccb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x558cfd93c870 0x558cfd93ccb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe01c066c10 msgr2=0x7fe01c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe01c066c10 0x7fe01c0690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0099b0 tx=0x7fe02c011000 comp rx=0 tx=0).stop 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 msgr2=0x558cfd953180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 --2- 172.21.15.53:0/3157353683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe024002260 0x558cfd953180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe034006460 tx=0x7fe034001d80 comp rx=0 tx=0).stop 2023-11-28T11:19:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe03a39c700 1 -- 172.21.15.53:0/3157353683 reap_dead start 2023-11-28T11:19:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 shutdown_connections 2023-11-28T11:19:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 >> 172.21.15.53:0/3157353683 conn(0x558cfd95b9d0 msgr2=0x558cfd9db640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 shutdown_connections 2023-11-28T11:19:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.754+0000 7fe0484e6080 1 -- 172.21.15.53:0/3157353683 wait complete. 2023-11-28T11:19:24.759 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2023-11-28T11:19:24.759 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:19:24.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.846+0000 7f8783c45080 1 Processor -- start 2023-11-28T11:19:24.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.846+0000 7f8783c45080 1 -- start start 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb250 0x560c78926aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c789bb620 0x560c78910100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7881dfe0 con 0x560c7892b920 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c789baa50 con 0x560c789bb250 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f8783c45080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c789bbb70 con 0x560c789bb620 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39518/0 (socket says 172.21.15.53:39518) 2023-11-28T11:19:24.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 -- 172.21.15.53:0/1930606906 learned_addr learned my addr 172.21.15.53:0/1930606906 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 -- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c789bb620 msgr2=0x560c78910100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c789bb620 0x560c78910100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 -- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb250 msgr2=0x560c78926aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb250 0x560c78926aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.847+0000 7f877539b700 1 -- 172.21.15.53:0/1930606906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768005d20 con 0x560c7892b920 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f877539b700 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8768000c00 tx=0x7f876800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a07a4cfef96b27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f87677fe700 1 -- 172.21.15.53:0/1930606906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800ae10 con 0x560c7892b920 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f87677fe700 1 -- 172.21.15.53:0/1930606906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800a220 con 0x560c7892b920 2023-11-28T11:19:24.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f87677fe700 1 -- 172.21.15.53:0/1930606906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768008a40 con 0x560c7892b920 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 -- 172.21.15.53:0/1930606906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 msgr2=0x560c789bf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8768000c00 tx=0x7f876800d000 comp rx=0 tx=0).stop 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 -- 172.21.15.53:0/1930606906 shutdown_connections 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c789bb620 0x560c78910100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c7892b920 0x560c789bf1d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 --2- 172.21.15.53:0/1930606906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb250 0x560c78926aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.848+0000 7f8783c45080 1 -- 172.21.15.53:0/1930606906 >> 172.21.15.53:0/1930606906 conn(0x560c7892f9d0 msgr2=0x560c78913560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.849+0000 7f8783c45080 1 -- 172.21.15.53:0/1930606906 shutdown_connections 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.849+0000 7f8783c45080 1 -- 172.21.15.53:0/1930606906 wait complete. 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.849+0000 7f8783c45080 1 Processor -- start 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 -- start start 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c7892b920 0x560c78928c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 0x560c789be240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb620 0x560c789b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c78929650 con 0x560c789bb250 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c789a8b50 con 0x560c789bb620 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.850+0000 7f8783c45080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c78929370 con 0x560c7892b920 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 0x560c789be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 0x560c789be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39528/0 (socket says 172.21.15.53:39528) 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 -- 172.21.15.53:0/1468949704 learned_addr learned my addr 172.21.15.53:0/1468949704 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f877539b700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c7892b920 0x560c78928c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8775b9c700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb620 0x560c789b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 -- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c7892b920 msgr2=0x560c78928c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c7892b920 0x560c78928c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 -- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb620 msgr2=0x560c789b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c789bb620 0x560c789b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 -- 172.21.15.53:0/1468949704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c788504f0 con 0x560c789bb250 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f877539b700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560c7892b920 0x560c78928c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f8774b9a700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 0x560c789be240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f876c008600 tx=0x7f876c0089c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.851+0000 7f87667fc700 1 -- 172.21.15.53:0/1468949704 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00f040 con 0x560c789bb250 2023-11-28T11:19:24.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.852+0000 7f87667fc700 1 -- 172.21.15.53:0/1468949704 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0127d0 con 0x560c789bb250 2023-11-28T11:19:24.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.852+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c78836120 con 0x560c789bb250 2023-11-28T11:19:24.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.852+0000 7f87667fc700 1 -- 172.21.15.53:0/1468949704 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c012b80 con 0x560c789bb250 2023-11-28T11:19:24.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.852+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c78829830 con 0x560c789bb250 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.854+0000 7f87667fc700 1 -- 172.21.15.53:0/1468949704 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f876c012ce0 con 0x560c789bb250 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f87667fc700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f87667fc700 1 -- 172.21.15.53:0/1468949704 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f876c084db0 con 0x560c789bb250 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f8783c45080 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x560c789be780 0x560c7890bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f877539b700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x560c7890c2d0 con 0x560c789be780 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.855+0000 7f8775b9c700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x560c789be780 0x560c7890bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.856+0000 7f877539b700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c789ad990 tx=0x7f8768014040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.856+0000 7f8775b9c700 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x560c789be780 0x560c7890bd90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8775b9c700 1 -- 172.21.15.53:0/1468949704 <== osd.5 v2:172.21.15.157:6808/132918 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f8770004520 con 0x560c789be780 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x560c789be780 msgr2=0x560c7890bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x560c789be780 0x560c7890bd90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 msgr2=0x7f87580690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c789ad990 tx=0x7f8768014040 comp rx=0 tx=0).stop 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 msgr2=0x560c789be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 --2- 172.21.15.53:0/1468949704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c789bb250 0x560c789be240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f876c008600 tx=0x7f876c0089c0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8775b9c700 1 -- 172.21.15.53:0/1468949704 reap_dead start 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 shutdown_connections 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 >> 172.21.15.53:0/1468949704 conn(0x560c7892f9d0 msgr2=0x560c78913560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 shutdown_connections 2023-11-28T11:19:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.857+0000 7f8783c45080 1 -- 172.21.15.53:0/1468949704 wait complete. 2023-11-28T11:19:24.862 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2023-11-28T11:19:24.862 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:19:24.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.949+0000 7f6194671080 1 Processor -- start 2023-11-28T11:19:24.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.949+0000 7f6194671080 1 -- start start 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 0x55967b834c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b791fe0 0x55967b837a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967b69bfe0 con 0x55967b761c30 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967b7913b0 con 0x55967b7648c0 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6194671080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55967b792910 con 0x55967b791fe0 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39542/0 (socket says 172.21.15.53:39542) 2023-11-28T11:19:24.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 -- 172.21.15.53:0/2460855433 learned_addr learned my addr 172.21.15.53:0/2460855433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f618679c700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b791fe0 0x55967b837a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f618579a700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 0x55967b834c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 -- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b791fe0 msgr2=0x55967b837a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b791fe0 0x55967b837a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 -- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 msgr2=0x55967b834c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 0x55967b834c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f6185f9b700 1 -- 172.21.15.53:0/2460855433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618000c3f0 con 0x55967b761c30 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.950+0000 7f618579a700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 0x55967b834c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6185f9b700 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f617800a7a0 tx=0x7f617800aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe427bff84c2675 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6177fff700 1 -- 172.21.15.53:0/2460855433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800f070 con 0x55967b761c30 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6177fff700 1 -- 172.21.15.53:0/2460855433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61780084c0 con 0x55967b761c30 2023-11-28T11:19:24.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6177fff700 1 -- 172.21.15.53:0/2460855433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178008850 con 0x55967b761c30 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6194671080 1 -- 172.21.15.53:0/2460855433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 msgr2=0x55967b8356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.951+0000 7f6194671080 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f617800a7a0 tx=0x7f617800aab0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 -- 172.21.15.53:0/2460855433 shutdown_connections 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b791fe0 0x55967b837a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b761c30 0x55967b8356e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 --2- 172.21.15.53:0/2460855433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b7648c0 0x55967b834c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 -- 172.21.15.53:0/2460855433 >> 172.21.15.53:0/2460855433 conn(0x55967b78e3e0 msgr2=0x55967b835e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 -- 172.21.15.53:0/2460855433 shutdown_connections 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 -- 172.21.15.53:0/2460855433 wait complete. 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.952+0000 7f6194671080 1 Processor -- start 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 -- start start 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b761c30 0x55967b7a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 0x55967b78f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b791fe0 0x55967b8d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967b7a69d0 con 0x55967b7648c0 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967b836d60 con 0x55967b791fe0 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.953+0000 7f6194671080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55967b8396c0 con 0x55967b761c30 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 0x55967b78f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f6185f9b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b761c30 0x55967b7a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 0x55967b78f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39544/0 (socket says 172.21.15.53:39544) 2023-11-28T11:19:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 -- 172.21.15.53:0/1543243868 learned_addr learned my addr 172.21.15.53:0/1543243868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618679c700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b791fe0 0x55967b8d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 -- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b761c30 msgr2=0x55967b7a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55967b761c30 0x55967b7a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 -- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b791fe0 msgr2=0x55967b8d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55967b791fe0 0x55967b8d2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.954+0000 7f618579a700 1 -- 172.21.15.53:0/1543243868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617800a420 con 0x55967b7648c0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f618579a700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 0x55967b78f920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f617c001ea0 tx=0x7f617c00a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f6176ffd700 1 -- 172.21.15.53:0/1543243868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c015030 con 0x55967b7648c0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f6176ffd700 1 -- 172.21.15.53:0/1543243868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c008040 con 0x55967b7648c0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f6176ffd700 1 -- 172.21.15.53:0/1543243868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00ae20 con 0x55967b7648c0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55967b6b4120 con 0x55967b7648c0 2023-11-28T11:19:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.955+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55967b6a7830 con 0x55967b7648c0 2023-11-28T11:19:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.957+0000 7f6176ffd700 1 -- 172.21.15.53:0/1543243868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f617c01b470 con 0x55967b7648c0 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.958+0000 7f6176ffd700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6168066c10 0x7f61680690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.958+0000 7f6176ffd700 1 -- 172.21.15.53:0/1543243868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f617c016070 con 0x55967b7648c0 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.958+0000 7f6194671080 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x55967b78fe60 0x55967b8d3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.958+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x55967b8329a0 con 0x55967b78fe60 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.958+0000 7f6185f9b700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6168066c10 0x7f61680690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.959+0000 7f618679c700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x55967b78fe60 0x55967b8d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.959+0000 7f618679c700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x55967b78fe60 0x55967b8d3460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.959+0000 7f6185f9b700 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6168066c10 0x7f61680690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6178017040 tx=0x7f6178012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f618679c700 1 -- 172.21.15.53:0/1543243868 <== osd.5 v2:172.21.15.157:6808/132918 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f618000a7a0 con 0x55967b78fe60 2023-11-28T11:19:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x55967b78fe60 msgr2=0x55967b8d3460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x55967b78fe60 0x55967b8d3460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6168066c10 msgr2=0x7f61680690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6168066c10 0x7f61680690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6178017040 tx=0x7f6178012000 comp rx=0 tx=0).stop 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 msgr2=0x55967b78f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 --2- 172.21.15.53:0/1543243868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55967b7648c0 0x55967b78f920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f617c001ea0 tx=0x7f617c00a4b0 comp rx=0 tx=0).stop 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f618679c700 1 -- 172.21.15.53:0/1543243868 reap_dead start 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 shutdown_connections 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 >> 172.21.15.53:0/1543243868 conn(0x55967b78e3e0 msgr2=0x55967b834c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 shutdown_connections 2023-11-28T11:19:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:24.960+0000 7f6194671080 1 -- 172.21.15.53:0/1543243868 wait complete. 2023-11-28T11:19:24.965 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2023-11-28T11:19:24.965 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:19:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.049+0000 7f7271a37080 1 Processor -- start 2023-11-28T11:19:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 -- start start 2023-11-28T11:19:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d087920 0x55856d11b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117620 0x55856d118550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856cf79fe0 con 0x55856d117250 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856d1153f0 con 0x55856d087920 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.050+0000 7f7271a37080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856d1190b0 con 0x55856d117620 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39552/0 (socket says 172.21.15.53:39552) 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 -- 172.21.15.53:0/3682665967 learned_addr learned my addr 172.21.15.53:0/3682665967 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726319b700 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d087920 0x55856d11b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726399c700 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117620 0x55856d118550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 -- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117620 msgr2=0x55856d118550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117620 0x55856d118550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 -- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d087920 msgr2=0x55856d11b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d087920 0x55856d11b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 -- 172.21.15.53:0/3682665967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254005de0 con 0x55856d117250 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.051+0000 7f726299a700 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f725800a3f0 tx=0x7f725800a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28d7357deebd1b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7261998700 1 -- 172.21.15.53:0/3682665967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800c830 con 0x55856d117250 2023-11-28T11:19:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7261998700 1 -- 172.21.15.53:0/3682665967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725800c990 con 0x55856d117250 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7261998700 1 -- 172.21.15.53:0/3682665967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800cd40 con 0x55856d117250 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 -- 172.21.15.53:0/3682665967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 msgr2=0x55856d118010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f725800a3f0 tx=0x7f725800a700 comp rx=0 tx=0).stop 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 -- 172.21.15.53:0/3682665967 shutdown_connections 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117620 0x55856d118550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d087920 0x55856d11b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 --2- 172.21.15.53:0/3682665967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d117250 0x55856d118010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 -- 172.21.15.53:0/3682665967 >> 172.21.15.53:0/3682665967 conn(0x55856d08b9d0 msgr2=0x55856d086900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 -- 172.21.15.53:0/3682665967 shutdown_connections 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.052+0000 7f7271a37080 1 -- 172.21.15.53:0/3682665967 wait complete. 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.053+0000 7f7271a37080 1 Processor -- start 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.053+0000 7f7271a37080 1 -- start start 2023-11-28T11:19:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 0x55856d10c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117250 0x55856d06c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d117620 0x55856d112ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856d06bd40 con 0x55856d087920 2023-11-28T11:19:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856d06b430 con 0x55856d117620 2023-11-28T11:19:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f7271a37080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856d111340 con 0x55856d117250 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f726319b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 0x55856d10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f726319b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 0x55856d10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39560/0 (socket says 172.21.15.53:39560) 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f726319b700 1 -- 172.21.15.53:0/520724728 learned_addr learned my addr 172.21.15.53:0/520724728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f726299a700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117250 0x55856d06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.054+0000 7f726399c700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d117620 0x55856d112ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 -- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117250 msgr2=0x55856d06c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117250 0x55856d06c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 -- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d117620 msgr2=0x55856d112ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d117620 0x55856d112ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 -- 172.21.15.53:0/520724728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258005ca0 con 0x55856d087920 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726399c700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55856d117620 0x55856d112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726299a700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55856d117250 0x55856d06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:25.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f726319b700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 0x55856d10c110 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f725400c010 tx=0x7f7254002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f7260996700 1 -- 172.21.15.53:0/520724728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401d920 con 0x55856d087920 2023-11-28T11:19:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f7260996700 1 -- 172.21.15.53:0/520724728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725401da80 con 0x55856d087920 2023-11-28T11:19:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55856cf92120 con 0x55856d087920 2023-11-28T11:19:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f7260996700 1 -- 172.21.15.53:0/520724728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401de30 con 0x55856d087920 2023-11-28T11:19:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.055+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856cf85830 con 0x55856d087920 2023-11-28T11:19:25.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:19:25.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.057+0000 7f7260996700 1 -- 172.21.15.53:0/520724728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7254012450 con 0x55856d087920 2023-11-28T11:19:25.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.058+0000 7f7260996700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7250066c10 0x7f72500690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.058+0000 7f7260996700 1 -- 172.21.15.53:0/520724728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7254086910 con 0x55856d087920 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.058+0000 7f7271a37080 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x55856d086900 0x55856d10eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.058+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x55856d06cf10 con 0x55856d086900 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.058+0000 7f726299a700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7250066c10 0x7f72500690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.059+0000 7f726399c700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x55856d086900 0x55856d10eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.059+0000 7f726299a700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7250066c10 0x7f72500690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55856d11aeb0 tx=0x7f7258018040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.059+0000 7f726399c700 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x55856d086900 0x55856d10eb00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f726399c700 1 -- 172.21.15.53:0/520724728 <== osd.7 v2:172.21.15.157:6824/132962 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f725c00d7f0 con 0x55856d086900 2023-11-28T11:19:25.072 INFO:teuthology.orchestra.run.smithi053.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:19:25.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x55856d086900 msgr2=0x55856d10eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x55856d086900 0x55856d10eb00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7250066c10 msgr2=0x7f72500690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7250066c10 0x7f72500690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55856d11aeb0 tx=0x7f7258018040 comp rx=0 tx=0).stop 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 msgr2=0x55856d10c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 --2- 172.21.15.53:0/520724728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55856d087920 0x55856d10c110 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f725400c010 tx=0x7f7254002fb0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f726399c700 1 -- 172.21.15.53:0/520724728 reap_dead start 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 shutdown_connections 2023-11-28T11:19:25.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 >> 172.21.15.53:0/520724728 conn(0x55856d08b9d0 msgr2=0x55856d0851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:25.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 shutdown_connections 2023-11-28T11:19:25.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.060+0000 7f7271a37080 1 -- 172.21.15.53:0/520724728 wait complete. 2023-11-28T11:19:25.074 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:19:25.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:19:25.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:19:25.285 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:19:25.285 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:25.285 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:25.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:19:25.330 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:19:25.330 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:25.377 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:19:25.378 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:25.378 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:25.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:19:25.424 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:19:25.424 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:25.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:19:25.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.671+0000 7fbd6b9ff700 1 Processor -- start 2023-11-28T11:19:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.671+0000 7fbd6b9ff700 1 -- start start 2023-11-28T11:19:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.672+0000 7fbd6b9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbd64161340 0x7fbd640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.672+0000 7fbd6b9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd640d2320 0x7fbd640cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.672+0000 7fbd6b9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640d1950 con 0x7fbd6415c9b0 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.672+0000 7fbd6b9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640b5800 con 0x7fbd640d2320 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.672+0000 7fbd6b9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640b5300 con 0x7fbd64161340 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd640d2320 0x7fbd640cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd68f9a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbd64161340 0x7fbd640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd640d2320 0x7fbd640cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39566/0 (socket says 172.21.15.53:39566) 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 -- 172.21.15.53:0/1489171736 learned_addr learned my addr 172.21.15.53:0/1489171736 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd6979b700 1 --1- 172.21.15.53:0/1489171736 >> v1:172.21.15.53:6789/0 conn(0x7fbd6415c9b0 0x7fbd640b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53022/0 (socket says 172.21.15.53:53022) 2023-11-28T11:19:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 -- 172.21.15.53:0/1489171736 >> v2:172.21.15.157:3300/0 conn(0x7fbd64161340 msgr2=0x7fbd640d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 --2- 172.21.15.53:0/1489171736 >> v2:172.21.15.157:3300/0 conn(0x7fbd64161340 0x7fbd640d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 -- 172.21.15.53:0/1489171736 >> v1:172.21.15.53:6789/0 conn(0x7fbd6415c9b0 legacy=0x7fbd640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 -- 172.21.15.53:0/1489171736 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640b5520 con 0x7fbd640d2320 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.673+0000 7fbd69f9c700 1 --2- 172.21.15.53:0/1489171736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd640d2320 0x7fbd640cd7c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd640685c0 tx=0x7fbd5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b0aebadc7d9cd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd5bfff700 1 -- 172.21.15.53:0/1489171736 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00e070 con 0x7fbd640d2320 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd5bfff700 1 -- 172.21.15.53:0/1489171736 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00b8d0 con 0x7fbd640d2320 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd5bfff700 1 -- 172.21.15.53:0/1489171736 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00bc80 con 0x7fbd640d2320 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/1489171736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd640d2320 msgr2=0x7fbd640cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/1489171736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd640d2320 0x7fbd640cd7c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd640685c0 tx=0x7fbd5c00b330 comp rx=0 tx=0).stop 2023-11-28T11:19:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/1489171736 shutdown_connections 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/1489171736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd640d2320 0x7fbd640cd7c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/1489171736 >> v2:172.21.15.157:3300/0 conn(0x7fbd64161340 0x7fbd640d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/1489171736 >> 172.21.15.53:0/1489171736 conn(0x7fbd64072600 msgr2=0x7fbd640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/1489171736 shutdown_connections 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.674+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/1489171736 wait complete. 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.676+0000 7fbd6b9ff700 1 Processor -- start 2023-11-28T11:19:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.676+0000 7fbd6b9ff700 1 -- start start 2023-11-28T11:19:25.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 0x7fbd640c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd64161340 0x7fbd64157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641e64c0 con 0x7fbd6415c9b0 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641e67a0 con 0x7fbd64161340 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.677+0000 7fbd6b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641e6a80 con 0x7fbd640d2320 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39576/0 (socket says 172.21.15.53:39576) 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 -- 172.21.15.53:0/3448838029 learned_addr learned my addr 172.21.15.53:0/3448838029 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd69f9c700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd64161340 0x7fbd64157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd6979b700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 0x7fbd640c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 -- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 msgr2=0x7fbd640c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 0x7fbd640c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 -- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd64161340 msgr2=0x7fbd64157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd64161340 0x7fbd64157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 -- 172.21.15.53:0/3448838029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009b30 con 0x7fbd6415c9b0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd6979b700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 0x7fbd640c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd68f9a700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e9b0 tx=0x7fbd5400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.678+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54009710 con 0x7fbd6415c9b0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.679+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5400c850 con 0x7fbd6415c9b0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.679+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400cbc0 con 0x7fbd6415c9b0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.679+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640d15e0 con 0x7fbd6415c9b0 2023-11-28T11:19:25.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.679+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd640d1300 con 0x7fbd6415c9b0 2023-11-28T11:19:25.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.680+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6415cd80 con 0x7fbd6415c9b0 2023-11-28T11:19:25.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.681+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd54019030 con 0x7fbd6415c9b0 2023-11-28T11:19:25.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.681+0000 7fbd59ffb700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 0x7fbd4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:25.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.681+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd54085ca0 con 0x7fbd6415c9b0 2023-11-28T11:19:25.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.682+0000 7fbd6979b700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 0x7fbd4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:25.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.683+0000 7fbd6979b700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 0x7fbd4806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000f60 tx=0x7fbd60007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:25.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.686+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd54052410 con 0x7fbd6415c9b0 2023-11-28T11:19:25.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.938+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd640745c0 con 0x7fbd6415c9b0 2023-11-28T11:19:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.939+0000 7fbd59ffb700 1 -- 172.21.15.53:0/3448838029 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fbd54017090 con 0x7fbd6415c9b0 2023-11-28T11:19:25.940 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:25.940 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 msgr2=0x7fbd4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 0x7fbd4806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000f60 tx=0x7fbd60007df0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 msgr2=0x7fbd6414dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e9b0 tx=0x7fbd5400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 shutdown_connections 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fbd480682f0 0x7fbd4806a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd640d2320 0x7fbd640c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd64161340 0x7fbd64157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 --2- 172.21.15.53:0/3448838029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd6415c9b0 0x7fbd6414dcd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.941+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 >> 172.21.15.53:0/3448838029 conn(0x7fbd64072600 msgr2=0x7fbd640b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.942+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 shutdown_connections 2023-11-28T11:19:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:25.942+0000 7fbd6b9ff700 1 -- 172.21.15.53:0/3448838029 wait complete. 2023-11-28T11:19:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:25.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:26.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.161+0000 7f39391ff700 1 Processor -- start 2023-11-28T11:19:26.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.161+0000 7f39391ff700 1 -- start start 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.162+0000 7f39391ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393415c9b0 0x7f39340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f39391ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39340d1950 con 0x7f3934161340 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f39391ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39340d24f0 con 0x7f39340cd7c0 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f39391ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340b5800 con 0x7f393415c9b0 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f392bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3934161340 0x7f39340d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53032/0 (socket says 172.21.15.53:53032) 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f392bfff700 1 -- 172.21.15.53:0/2196189417 learned_addr learned my addr 172.21.15.53:0/2196189417 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:26.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f3932d9d700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393415c9b0 0x7f39340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481067578 0 0) 0x7f39340d1950 con 0x7f3934161340 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39200035b0 con 0x7f3934161340 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510488808 0 0) 0x7f39340d24f0 con 0x7f39340cd7c0 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.163+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39340d1950 con 0x7f39340cd7c0 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775382437 0 0) 0x7f39200035b0 con 0x7f3934161340 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393415c9b0 msgr2=0x7f39340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393415c9b0 0x7f39340b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 >> v1:172.21.15.157:6789/0 conn(0x7f39340cd7c0 legacy=0x7f39340b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340b5520 con 0x7f3934161340 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f391c001670 con 0x7f3934161340 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 >> v1:172.21.15.53:6789/0 conn(0x7f3934161340 legacy=0x7f39340d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3920003bc0 0x7f3920003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f3920004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920005200 0x7f392000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340b5800 con 0x7f3920004850 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392000c0e0 con 0x7f3920005200 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392000c270 con 0x7f3920003bc0 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f3932d9d700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393415c9b0 0x7f39340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f3920004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393359e700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920005200 0x7f392000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f3932d9d700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3920003bc0 0x7f3920003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 -- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3920003bc0 msgr2=0x7f3920003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3920003bc0 0x7f3920003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 -- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920005200 msgr2=0x7f392000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920005200 0x7f392000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f392bfff700 1 -- 172.21.15.53:0/2196189417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200035b0 con 0x7f3920004850 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.164+0000 7f393359e700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3920005200 0x7f392000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f392bfff700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f3920004cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c160 tx=0x7f3920005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6cd7f715fc9ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c00cca0 con 0x7f3920004850 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c00ce20 con 0x7f3920004850 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f393259c700 1 -- 172.21.15.53:0/2196189417 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c005ba0 con 0x7f3920004850 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f39391ff700 1 -- 172.21.15.53:0/2196189417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 msgr2=0x7f3920004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f39391ff700 1 --2- 172.21.15.53:0/2196189417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f3920004cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c160 tx=0x7f3920005880 comp rx=0 tx=0).stop 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f393359e700 1 -- 172.21.15.53:0/2196189417 reap_dead start 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f39391ff700 1 -- 172.21.15.53:0/2196189417 shutdown_connections 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f39391ff700 1 -- 172.21.15.53:0/2196189417 >> 172.21.15.53:0/2196189417 conn(0x7f3934072600 msgr2=0x7f39340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.165+0000 7f39391ff700 1 -- 172.21.15.53:0/2196189417 shutdown_connections 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.166+0000 7f39391ff700 1 -- 172.21.15.53:0/2196189417 wait complete. 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.167+0000 7f39391ff700 1 Processor -- start 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.167+0000 7f39391ff700 1 -- start start 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 0x7f3934156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3934157430 0x7f39341578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340c71b0 con 0x7f3920004850 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340b8350 con 0x7f3934157430 2023-11-28T11:19:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.168+0000 7f39391ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340b8630 con 0x7f39341561e0 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39590/0 (socket says 172.21.15.53:39590) 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 -- 172.21.15.53:0/1121838039 learned_addr learned my addr 172.21.15.53:0/1121838039 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f392bfff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 0x7f3934156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f393359e700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3934157430 0x7f39341578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 -- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 msgr2=0x7f3934156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 0x7f3934156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 -- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3934157430 msgr2=0x7f39341578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3934157430 0x7f39341578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 -- 172.21.15.53:0/1121838039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340d15e0 con 0x7f3920004850 2023-11-28T11:19:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f392bfff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 0x7f3934156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f3932d9d700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3924011800 tx=0x7f3924011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.169+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924015870 con 0x7f3920004850 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.170+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39240159d0 con 0x7f3920004850 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.170+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924015d80 con 0x7f3920004850 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.170+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39340d1300 con 0x7f3920004850 2023-11-28T11:19:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.170+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393415cd80 con 0x7f3920004850 2023-11-28T11:19:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.171+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39340745c0 con 0x7f3920004850 2023-11-28T11:19:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.172+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3924012040 con 0x7f3920004850 2023-11-28T11:19:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.173+0000 7f392b7fe700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 0x7f391406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.173+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3924085ff0 con 0x7f3920004850 2023-11-28T11:19:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.174+0000 7f392bfff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 0x7f391406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.174+0000 7f392bfff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 0x7f391406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f391c015040 tx=0x7f391c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.177+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3924052780 con 0x7f3920004850 2023-11-28T11:19:26.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.380+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39340b8940 con 0x7f3920004850 2023-11-28T11:19:26.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.382+0000 7f392b7fe700 1 -- 172.21.15.53:0/1121838039 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f3924022090 con 0x7f3920004850 2023-11-28T11:19:26.383 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:00.492727+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:26.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 msgr2=0x7f391406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 0x7f391406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f391c015040 tx=0x7f391c007000 comp rx=0 tx=0).stop 2023-11-28T11:19:26.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 msgr2=0x7f39341558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3924011800 tx=0x7f3924011bc0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 shutdown_connections 2023-11-28T11:19:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39140682f0 0x7f391406a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3934157430 0x7f39341578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3920004850 0x7f39341558e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 --2- 172.21.15.53:0/1121838039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39341561e0 0x7f3934156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 >> 172.21.15.53:0/1121838039 conn(0x7f3934072600 msgr2=0x7f393415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 shutdown_connections 2023-11-28T11:19:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.384+0000 7f39391ff700 1 -- 172.21.15.53:0/1121838039 wait complete. 2023-11-28T11:19:26.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:18:57.143723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:18:58.463907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:19:26.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:19:26.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.610+0000 7fddb5bff700 1 Processor -- start 2023-11-28T11:19:26.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.610+0000 7fddb5bff700 1 -- start start 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddb5bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 0x7fddb00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddb5bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb00d1950 con 0x7fddb00cd7c0 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddb5bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb00d24f0 con 0x7fddb0161340 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddb5bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b5800 con 0x7fddb015c9b0 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddaffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fddb00cd7c0 0x7fddb00b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53036/0 (socket says 172.21.15.53:53036) 2023-11-28T11:19:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.611+0000 7fddaffff700 1 -- 172.21.15.53:0/1489731094 learned_addr learned my addr 172.21.15.53:0/1489731094 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddaf7fe700 1 --2- 172.21.15.53:0/1489731094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 0x7fddb00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151886495 0 0) 0x7fddb00d1950 con 0x7fddb00cd7c0 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd980035b0 con 0x7fddb00cd7c0 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305459395 0 0) 0x7fddb00d24f0 con 0x7fddb0161340 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb00d1950 con 0x7fddb0161340 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553544499 0 0) 0x7fdd980035b0 con 0x7fddb00cd7c0 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddaf7fe700 1 -- 172.21.15.53:0/1489731094 >> v1:172.21.15.157:6789/0 conn(0x7fddb0161340 legacy=0x7fddb00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddaf7fe700 1 -- 172.21.15.53:0/1489731094 >> v1:172.21.15.53:6789/0 conn(0x7fddb00cd7c0 legacy=0x7fddb00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.612+0000 7fddaf7fe700 1 -- 172.21.15.53:0/1489731094 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00b5520 con 0x7fddb015c9b0 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddaf7fe700 1 --2- 172.21.15.53:0/1489731094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 0x7fddb00b4bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009870 tx=0x7fdda0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df09e018869756a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda80019f0 con 0x7fddb015c9b0 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda0004030 con 0x7fddb015c9b0 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddae7fc700 1 -- 172.21.15.53:0/1489731094 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda00043c0 con 0x7fddb015c9b0 2023-11-28T11:19:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddb5bff700 1 -- 172.21.15.53:0/1489731094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 msgr2=0x7fddb00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1489731094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 0x7fddb00b4bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009870 tx=0x7fdda0009b80 comp rx=0 tx=0).stop 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddb5bff700 1 -- 172.21.15.53:0/1489731094 shutdown_connections 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1489731094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb015c9b0 0x7fddb00b4bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.613+0000 7fddb5bff700 1 -- 172.21.15.53:0/1489731094 >> 172.21.15.53:0/1489731094 conn(0x7fddb0072600 msgr2=0x7fddb00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.614+0000 7fddb5bff700 1 -- 172.21.15.53:0/1489731094 shutdown_connections 2023-11-28T11:19:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.614+0000 7fddb5bff700 1 -- 172.21.15.53:0/1489731094 wait complete. 2023-11-28T11:19:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.615+0000 7fddb5bff700 1 Processor -- start 2023-11-28T11:19:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.615+0000 7fddb5bff700 1 -- start start 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb00cd7c0 0x7fddb014d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb015c9b0 0x7fddb00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b77d0 con 0x7fddb0161340 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb014db60 con 0x7fddb015c9b0 2023-11-28T11:19:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.616+0000 7fddb5bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b85d0 con 0x7fddb00cd7c0 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39596/0 (socket says 172.21.15.53:39596) 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 -- 172.21.15.53:0/1144548525 learned_addr learned my addr 172.21.15.53:0/1144548525 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaeffd700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb015c9b0 0x7fddb00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaf7fe700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb00cd7c0 0x7fddb014d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 -- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb00cd7c0 msgr2=0x7fddb014d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb00cd7c0 0x7fddb014d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 -- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb015c9b0 msgr2=0x7fddb00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb015c9b0 0x7fddb00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 -- 172.21.15.53:0/1144548525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00094a0 con 0x7fddb0161340 2023-11-28T11:19:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.617+0000 7fddaffff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdda800cbe0 tx=0x7fdda800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.618+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda800abc0 con 0x7fddb0161340 2023-11-28T11:19:26.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.618+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda800ad20 con 0x7fddb0161340 2023-11-28T11:19:26.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.618+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8007310 con 0x7fddb0161340 2023-11-28T11:19:26.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.618+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00d15e0 con 0x7fddb0161340 2023-11-28T11:19:26.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.618+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb00d1300 con 0x7fddb0161340 2023-11-28T11:19:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.619+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb00b5130 con 0x7fddb0161340 2023-11-28T11:19:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.620+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdda8010070 con 0x7fddb0161340 2023-11-28T11:19:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.620+0000 7fdd97fff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 0x7fdd9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.621+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdda8085780 con 0x7fddb0161340 2023-11-28T11:19:26.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.621+0000 7fddaf7fe700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 0x7fdd9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:26.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.622+0000 7fddaf7fe700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 0x7fdd9c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdda00090b0 tx=0x7fdda0013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:26.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.624+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdda8051ef0 con 0x7fddb0161340 2023-11-28T11:19:26.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.827+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddb00745c0 con 0x7fddb0161340 2023-11-28T11:19:26.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.827+0000 7fdd97fff700 1 -- 172.21.15.53:0/1144548525 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdda8018020 con 0x7fddb0161340 2023-11-28T11:19:26.829 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:19:26.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 msgr2=0x7fdd9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 0x7fdd9c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdda00090b0 tx=0x7fdda0013040 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 msgr2=0x7fddb00cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdda800cbe0 tx=0x7fdda800a060 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 shutdown_connections 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd9c0682f0 0x7fdd9c06a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb00cd7c0 0x7fddb014d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb0161340 0x7fddb00cae80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 --2- 172.21.15.53:0/1144548525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb015c9b0 0x7fddb00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 >> 172.21.15.53:0/1144548525 conn(0x7fddb0072600 msgr2=0x7fddb015a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 shutdown_connections 2023-11-28T11:19:26.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:26.830+0000 7fddb5bff700 1 -- 172.21.15.53:0/1144548525 wait complete. 2023-11-28T11:19:26.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:19:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.053+0000 7ff6d77ff700 1 Processor -- start 2023-11-28T11:19:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.053+0000 7ff6d77ff700 1 -- start start 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d015c9b0 0x7ff6d00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6d0161340 0x7ff6d00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6d00d2320 0x7ff6d00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00b5800 con 0x7ff6d0161340 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00d1950 con 0x7ff6d00d2320 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00b5520 con 0x7ff6d015c9b0 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d4d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6d0161340 0x7ff6d00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d4d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6d0161340 0x7ff6d00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39612/0 (socket says 172.21.15.53:39612) 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.054+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/961179265 learned_addr learned my addr 172.21.15.53:0/961179265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d559b700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d015c9b0 0x7ff6d00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d5d9c700 1 --2- 172.21.15.53:0/961179265 >> v2:172.21.15.157:3300/0 conn(0x7ff6d00d2320 0x7ff6d00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/961179265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d015c9b0 msgr2=0x7ff6d00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d015c9b0 0x7ff6d00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/961179265 >> v2:172.21.15.157:3300/0 conn(0x7ff6d00d2320 msgr2=0x7ff6d00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/961179265 >> v2:172.21.15.157:3300/0 conn(0x7ff6d00d2320 0x7ff6d00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/961179265 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00d1300 con 0x7ff6d0161340 2023-11-28T11:19:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0161340 0x7ff6d00d1de0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6d008c4d0 tx=0x7ff6c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a5301bb761508b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6c7fff700 1 -- 172.21.15.53:0/961179265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000b870 con 0x7ff6d0161340 2023-11-28T11:19:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6c7fff700 1 -- 172.21.15.53:0/961179265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c000b9d0 con 0x7ff6d0161340 2023-11-28T11:19:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.055+0000 7ff6c7fff700 1 -- 172.21.15.53:0/961179265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000bd60 con 0x7ff6d0161340 2023-11-28T11:19:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 -- 172.21.15.53:0/961179265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0161340 msgr2=0x7ff6d00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0161340 0x7ff6d00d1de0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6d008c4d0 tx=0x7ff6c0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 -- 172.21.15.53:0/961179265 shutdown_connections 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d015c9b0 0x7ff6d00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/961179265 >> v2:172.21.15.157:3300/0 conn(0x7ff6d00d2320 0x7ff6d00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/961179265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0161340 0x7ff6d00d1de0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 -- 172.21.15.53:0/961179265 >> 172.21.15.53:0/961179265 conn(0x7ff6d0072600 msgr2=0x7ff6d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 -- 172.21.15.53:0/961179265 shutdown_connections 2023-11-28T11:19:27.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.056+0000 7ff6d77ff700 1 -- 172.21.15.53:0/961179265 wait complete. 2023-11-28T11:19:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.058+0000 7ff6d77ff700 1 Processor -- start 2023-11-28T11:19:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.058+0000 7ff6d77ff700 1 -- start start 2023-11-28T11:19:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 0x7ff6d00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d0161340 0x7ff6d015ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00c7bf0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00cd360 con 0x7ff6d00d2320 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d77ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00cd640 con 0x7ff6d0161340 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d4d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d4d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39614/0 (socket says 172.21.15.53:39614) 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.059+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/895634531 learned_addr learned my addr 172.21.15.53:0/895634531 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d5d9c700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d0161340 0x7ff6d015ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d559b700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 0x7ff6d00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d0161340 msgr2=0x7ff6d015ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d0161340 0x7ff6d015ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 msgr2=0x7ff6d00ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 0x7ff6d00ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 -- 172.21.15.53:0/895634531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0009550 con 0x7ff6d015c9b0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d559b700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 0x7ff6d00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d4d9a700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0004540 con 0x7ff6d015c9b0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c00036a0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0003a50 con 0x7ff6d015c9b0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.060+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d00b5130 con 0x7ff6d015c9b0 2023-11-28T11:19:27.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.061+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d00d15e0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.062+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d00b52c0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.062+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6c000f050 con 0x7ff6d015c9b0 2023-11-28T11:19:27.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.063+0000 7ff6c5ffb700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 0x7ff6b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.063+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6c0087660 con 0x7ff6d015c9b0 2023-11-28T11:19:27.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.064+0000 7ff6d559b700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 0x7ff6b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.065+0000 7ff6d559b700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 0x7ff6b406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0079d0 tx=0x7ff6cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.068+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6c0053dd0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.267+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6d00745c0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.268+0000 7ff6c5ffb700 1 -- 172.21.15.53:0/895634531 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6c00573f0 con 0x7ff6d015c9b0 2023-11-28T11:19:27.269 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:19:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 msgr2=0x7ff6b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 0x7ff6b406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0079d0 tx=0x7ff6cc008040 comp rx=0 tx=0).stop 2023-11-28T11:19:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 msgr2=0x7ff6d00c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c0004750 comp rx=0 tx=0).stop 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 shutdown_connections 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff6b40682f0 0x7ff6b406a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d0161340 0x7ff6d015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d00d2320 0x7ff6d00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 --2- 172.21.15.53:0/895634531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d015c9b0 0x7ff6d00c76b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 >> 172.21.15.53:0/895634531 conn(0x7ff6d0072600 msgr2=0x7ff6d015a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 shutdown_connections 2023-11-28T11:19:27.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.270+0000 7ff6d77ff700 1 -- 172.21.15.53:0/895634531 wait complete. 2023-11-28T11:19:27.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:19:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.493+0000 7fa2eddff700 1 Processor -- start 2023-11-28T11:19:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.493+0000 7fa2eddff700 1 -- start start 2023-11-28T11:19:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2eddff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2e815c9b0 0x7fa2e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2eddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 0x7fa2e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2eddff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80d1950 con 0x7fa2e80d2320 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2eddff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80b5800 con 0x7fa2e815c9b0 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2eddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80b5520 con 0x7fa2e8161340 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2e7fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa2e80d2320 0x7fa2e80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53040/0 (socket says 172.21.15.53:53040) 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2e7fff700 1 -- 172.21.15.53:0/3078070242 learned_addr learned my addr 172.21.15.53:0/3078070242 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.494+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/3078070242 >> v2:172.21.15.157:3300/0 conn(0x7fa2e815c9b0 0x7fa2e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e6ffd700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 0x7fa2e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e67fc700 1 -- 172.21.15.53:0/3078070242 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700076245 0 0) 0x7fa2e80d1950 con 0x7fa2e80d2320 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e67fc700 1 -- 172.21.15.53:0/3078070242 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d00035b0 con 0x7fa2e80d2320 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e77fe700 1 -- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 msgr2=0x7fa2e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 0x7fa2e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e77fe700 1 -- 172.21.15.53:0/3078070242 >> v1:172.21.15.53:6789/0 conn(0x7fa2e80d2320 legacy=0x7fa2e80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e77fe700 1 -- 172.21.15.53:0/3078070242 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80d1300 con 0x7fa2e815c9b0 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e6ffd700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 0x7fa2e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.495+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e815c9b0 0x7fa2e80b4bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009840 tx=0x7fa2d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a161f0eac49a5603 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2e67fc700 1 -- 172.21.15.53:0/3078070242 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8011830 con 0x7fa2e815c9b0 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2e67fc700 1 -- 172.21.15.53:0/3078070242 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8011990 con 0x7fa2e815c9b0 2023-11-28T11:19:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2e67fc700 1 -- 172.21.15.53:0/3078070242 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8011d20 con 0x7fa2e815c9b0 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 -- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e815c9b0 msgr2=0x7fa2e80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e815c9b0 0x7fa2e80b4bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009840 tx=0x7fa2d8009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 -- 172.21.15.53:0/3078070242 shutdown_connections 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e815c9b0 0x7fa2e80b4bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 --2- 172.21.15.53:0/3078070242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e8161340 0x7fa2e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 -- 172.21.15.53:0/3078070242 >> 172.21.15.53:0/3078070242 conn(0x7fa2e8072600 msgr2=0x7fa2e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 -- 172.21.15.53:0/3078070242 shutdown_connections 2023-11-28T11:19:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.496+0000 7fa2eddff700 1 -- 172.21.15.53:0/3078070242 wait complete. 2023-11-28T11:19:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.498+0000 7fa2eddff700 1 Processor -- start 2023-11-28T11:19:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.498+0000 7fa2eddff700 1 -- start start 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e80d2320 0x7fa2e8146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8161340 0x7fa2e8152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8146840 con 0x7fa2e815c9b0 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8154250 con 0x7fa2e8161340 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2eddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8153640 con 0x7fa2e80d2320 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39622/0 (socket says 172.21.15.53:39622) 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2e6ffd700 1 -- 172.21.15.53:0/1885683987 learned_addr learned my addr 172.21.15.53:0/1885683987 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e80d2320 0x7fa2e8146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.499+0000 7fa2e7fff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8161340 0x7fa2e8152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 -- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e80d2320 msgr2=0x7fa2e8146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e80d2320 0x7fa2e8146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 -- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8161340 msgr2=0x7fa2e8152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8161340 0x7fa2e8152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 -- 172.21.15.53:0/1885683987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0094a0 con 0x7fa2e815c9b0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2e6ffd700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ebe0 tx=0x7fa2dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009800 con 0x7fa2e815c9b0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc00c910 con 0x7fa2e815c9b0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80094a0 con 0x7fa2e815c9b0 2023-11-28T11:19:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc00cc80 con 0x7fa2e815c9b0 2023-11-28T11:19:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.500+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e80b5130 con 0x7fa2e815c9b0 2023-11-28T11:19:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.501+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e80d15e0 con 0x7fa2e815c9b0 2023-11-28T11:19:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.502+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2dc019030 con 0x7fa2e815c9b0 2023-11-28T11:19:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.503+0000 7fa2cffff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 0x7fa2d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.503+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2dc085db0 con 0x7fa2e815c9b0 2023-11-28T11:19:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.504+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 0x7fa2d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.505+0000 7fa2e77fe700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 0x7fa2d406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8000c00 tx=0x7fa2d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.507+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2dc08a070 con 0x7fa2e815c9b0 2023-11-28T11:19:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.710+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2e80b52c0 con 0x7fa2e815c9b0 2023-11-28T11:19:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.710+0000 7fa2cffff700 1 -- 172.21.15.53:0/1885683987 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2dc017020 con 0x7fa2e815c9b0 2023-11-28T11:19:27.711 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.712+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 msgr2=0x7fa2d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.712+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 0x7fa2d406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8000c00 tx=0x7fa2d8008030 comp rx=0 tx=0).stop 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.712+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 msgr2=0x7fa2e8151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.712+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ebe0 tx=0x7fa2dc00eef0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 shutdown_connections 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa2d40682f0 0x7fa2d406a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e80d2320 0x7fa2e8146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8161340 0x7fa2e8152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 --2- 172.21.15.53:0/1885683987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2e815c9b0 0x7fa2e8151de0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 >> 172.21.15.53:0/1885683987 conn(0x7fa2e8072600 msgr2=0x7fa2e815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 shutdown_connections 2023-11-28T11:19:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.713+0000 7fa2eddff700 1 -- 172.21.15.53:0/1885683987 wait complete. 2023-11-28T11:19:27.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:19:27.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.938+0000 7fe370bff700 1 Processor -- start 2023-11-28T11:19:27.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.938+0000 7fe370bff700 1 -- start start 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.939+0000 7fe370bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe370bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0ccc00 con 0x7fe36c0cdc80 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe370bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0ccec0 con 0x7fe36c159860 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe370bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0cd1a0 con 0x7fe36c159490 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36ad9d700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe36c0cdc80 0x7fe36c06fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53054/0 (socket says 172.21.15.53:53054) 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36ad9d700 1 -- 172.21.15.53:0/2579490265 learned_addr learned my addr 172.21.15.53:0/2579490265 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36a59c700 1 --2- 172.21.15.53:0/2579490265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829396959 0 0) 0x7fe36c0ccc00 con 0x7fe36c0cdc80 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3540035b0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034592478 0 0) 0x7fe36c0ccec0 con 0x7fe36c159860 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.940+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe36c0ccc00 con 0x7fe36c159860 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36a59c700 1 -- 172.21.15.53:0/2579490265 >> v1:172.21.15.157:6789/0 conn(0x7fe36c159860 legacy=0x7fe36c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36a59c700 1 -- 172.21.15.53:0/2579490265 >> v1:172.21.15.53:6789/0 conn(0x7fe36c0cdc80 legacy=0x7fe36c06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36a59c700 1 -- 172.21.15.53:0/2579490265 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0cd480 con 0x7fe36c159490 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36a59c700 1 --2- 172.21.15.53:0/2579490265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c0b0670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc6f6462b8a068c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011830 con 0x7fe36c159490 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c011990 con 0x7fe36c159490 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.941+0000 7fe36959a700 1 -- 172.21.15.53:0/2579490265 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011d20 con 0x7fe36c159490 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 -- 172.21.15.53:0/2579490265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 msgr2=0x7fe36c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 --2- 172.21.15.53:0/2579490265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c0b0670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c009b80 comp rx=0 tx=0).stop 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 -- 172.21.15.53:0/2579490265 shutdown_connections 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 --2- 172.21.15.53:0/2579490265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c0b0670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 -- 172.21.15.53:0/2579490265 >> 172.21.15.53:0/2579490265 conn(0x7fe36c0cb670 msgr2=0x7fe36c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 -- 172.21.15.53:0/2579490265 shutdown_connections 2023-11-28T11:19:27.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.942+0000 7fe370bff700 1 -- 172.21.15.53:0/2579490265 wait complete. 2023-11-28T11:19:27.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.944+0000 7fe370bff700 1 Processor -- start 2023-11-28T11:19:27.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.944+0000 7fe370bff700 1 -- start start 2023-11-28T11:19:27.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe36c06cd40 0x7fe36c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1569b0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c06d700 con 0x7fe36c06cd40 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe370bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c155d30 con 0x7fe36c159490 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39636/0 (socket says 172.21.15.53:39636) 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 -- 172.21.15.53:0/4202395674 learned_addr learned my addr 172.21.15.53:0/4202395674 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36ad9d700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe36c06cd40 0x7fe36c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe369d9b700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 -- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 msgr2=0x7fe36c1f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c1f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 -- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe36c06cd40 msgr2=0x7fe36c06d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe36c06cd40 0x7fe36c06d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.945+0000 7fe36a59c700 1 -- 172.21.15.53:0/4202395674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0094a0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe36a59c700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0090b0 tx=0x7fe35c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c01a6e0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c01a840 con 0x7fe36c0cdc80 2023-11-28T11:19:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0b0bb0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c01abb0 con 0x7fe36c0cdc80 2023-11-28T11:19:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.946+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0cd760 con 0x7fe36c0cdc80 2023-11-28T11:19:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.947+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c0b0d40 con 0x7fe36c0cdc80 2023-11-28T11:19:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.948+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe35c00f050 con 0x7fe36c0cdc80 2023-11-28T11:19:27.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.949+0000 7fe35affd700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 0x7fe34c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:27.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.949+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe35c086a30 con 0x7fe36c0cdc80 2023-11-28T11:19:27.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.950+0000 7fe369d9b700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 0x7fe34c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:27.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.950+0000 7fe369d9b700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 0x7fe34c06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe36c142a00 tx=0x7fe360009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:27.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:27.954+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe35c053250 con 0x7fe36c0cdc80 2023-11-28T11:19:28.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.155+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe36c156c20 con 0x7fe36c0cdc80 2023-11-28T11:19:28.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.155+0000 7fe35affd700 1 -- 172.21.15.53:0/4202395674 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe35c029020 con 0x7fe36c0cdc80 2023-11-28T11:19:28.156 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 msgr2=0x7fe34c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 0x7fe34c06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe36c142a00 tx=0x7fe360009040 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 msgr2=0x7fe36c156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0090b0 tx=0x7fe35c004750 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 shutdown_connections 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe34c0682f0 0x7fe34c06a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe36c06cd40 0x7fe36c06d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe36c0cdc80 0x7fe36c156dd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 --2- 172.21.15.53:0/4202395674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe36c159490 0x7fe36c1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 >> 172.21.15.53:0/4202395674 conn(0x7fe36c0cb670 msgr2=0x7fe36c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:28.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.157+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 shutdown_connections 2023-11-28T11:19:28.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.158+0000 7fe370bff700 1 -- 172.21.15.53:0/4202395674 wait complete. 2023-11-28T11:19:28.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:28.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.377+0000 7f22287ff700 1 Processor -- start 2023-11-28T11:19:28.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.377+0000 7f22287ff700 1 -- start start 2023-11-28T11:19:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.378+0000 7f22287ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f22200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.378+0000 7f22287ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22200d1950 con 0x7f222015c9b0 2023-11-28T11:19:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.378+0000 7f22287ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22200b5520 con 0x7f22200d2320 2023-11-28T11:19:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.378+0000 7f22287ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22200b5800 con 0x7f2220161340 2023-11-28T11:19:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f222659b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f222015c9b0 0x7f22200b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53068/0 (socket says 172.21.15.53:53068) 2023-11-28T11:19:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f222659b700 1 -- 172.21.15.53:0/1637300771 learned_addr learned my addr 172.21.15.53:0/1637300771 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571671745 0 0) 0x7f22200d1950 con 0x7f222015c9b0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225d9a700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f22200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f220c0035b0 con 0x7f222015c9b0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246952137 0 0) 0x7f220c0035b0 con 0x7f222015c9b0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 msgr2=0x7f22200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f22200d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 >> v1:172.21.15.157:6789/0 conn(0x7f22200d2320 legacy=0x7f22200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200d1300 con 0x7f222015c9b0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f221c001670 con 0x7f222015c9b0 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.379+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 >> v1:172.21.15.53:6789/0 conn(0x7f222015c9b0 legacy=0x7f22200b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f220c003a30 0x7f220c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f220c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c004d90 0x7f220c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22200b5800 con 0x7f220c003a30 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c00bb20 con 0x7f220c004d90 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c00bce0 con 0x7f2220161340 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f220c003a30 0x7f220c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2225d9a700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f220c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2226d9c700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c004d90 0x7f220c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 -- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 msgr2=0x7f220c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2220161340 0x7f220c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 -- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c004d90 msgr2=0x7f220c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c004d90 0x7f220c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 -- 172.21.15.53:0/1637300771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0035b0 con 0x7f220c003a30 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f2226d9c700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c004d90 0x7f220c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.380+0000 7f222659b700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f220c003a30 0x7f220c003e40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c240 tx=0x7f220c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4f677ce4e941e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c007c10 con 0x7f220c003a30 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c00ce00 con 0x7f220c003a30 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f2225599700 1 -- 172.21.15.53:0/1637300771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c005ba0 con 0x7f220c003a30 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 -- 172.21.15.53:0/1637300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f220c003a30 msgr2=0x7f220c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 --2- 172.21.15.53:0/1637300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f220c003a30 0x7f220c003e40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c240 tx=0x7f220c005880 comp rx=0 tx=0).stop 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f2226d9c700 1 -- 172.21.15.53:0/1637300771 reap_dead start 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 -- 172.21.15.53:0/1637300771 shutdown_connections 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 -- 172.21.15.53:0/1637300771 >> 172.21.15.53:0/1637300771 conn(0x7f2220072600 msgr2=0x7f22200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 -- 172.21.15.53:0/1637300771 shutdown_connections 2023-11-28T11:19:28.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.381+0000 7f22287ff700 1 -- 172.21.15.53:0/1637300771 wait complete. 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.383+0000 7f22287ff700 1 Processor -- start 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.383+0000 7f22287ff700 1 -- start start 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 0x7f2220157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 0x7f222014a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201e84d0 con 0x7f22201e7b20 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f3ad0 con 0x7f220c003a30 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f22287ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f2f50 con 0x7f22201f2940 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f2226d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f2226d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39664/0 (socket says 172.21.15.53:39664) 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.384+0000 7f2226d9c700 1 -- 172.21.15.53:0/146027567 learned_addr learned my addr 172.21.15.53:0/146027567 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2225d9a700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 0x7f222014a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f222659b700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 0x7f2220157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 -- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 msgr2=0x7f222014a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 0x7f222014a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 -- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 msgr2=0x7f2220157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 0x7f2220157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 -- 172.21.15.53:0/146027567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222015cd80 con 0x7f22201e7b20 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f222659b700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 0x7f2220157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2225d9a700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 0x7f222014a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2226d9c700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f222008bfd0 tx=0x7f221800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218003c10 con 0x7f22201e7b20 2023-11-28T11:19:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2218003d90 con 0x7f22201e7b20 2023-11-28T11:19:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218005310 con 0x7f22201e7b20 2023-11-28T11:19:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.385+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22200d15e0 con 0x7f22201e7b20 2023-11-28T11:19:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.386+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222015cf10 con 0x7f22201e7b20 2023-11-28T11:19:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.387+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22201f2d10 con 0x7f22201e7b20 2023-11-28T11:19:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.388+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2218005470 con 0x7f22201e7b20 2023-11-28T11:19:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.388+0000 7f2216ffd700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 0x7f220406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.388+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2218085dd0 con 0x7f22201e7b20 2023-11-28T11:19:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.389+0000 7f222659b700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 0x7f220406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.390+0000 7f222659b700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 0x7f220406a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221c001930 tx=0x7f221c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.392+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22201f2d10 con 0x7f22201e7b20 2023-11-28T11:19:28.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.637+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22201f2d10 con 0x7f22201e7b20 2023-11-28T11:19:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.638+0000 7f2216ffd700 1 -- 172.21.15.53:0/146027567 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f22201f2d10 con 0x7f22201e7b20 2023-11-28T11:19:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:28.639 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 msgr2=0x7f220406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 0x7f220406a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221c001930 tx=0x7f221c007000 comp rx=0 tx=0).stop 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 msgr2=0x7f22201e7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f222008bfd0 tx=0x7f221800ee00 comp rx=0 tx=0).stop 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 shutdown_connections 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f22040682f0 0x7f220406a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22201e7b20 0x7f22201e7f90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f220c003a30 0x7f2220157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 --2- 172.21.15.53:0/146027567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22201f2940 0x7f222014a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 >> 172.21.15.53:0/146027567 conn(0x7f2220072600 msgr2=0x7f22200b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 shutdown_connections 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.640+0000 7f22287ff700 1 -- 172.21.15.53:0/146027567 wait complete. 2023-11-28T11:19:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:28.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.861+0000 7ff21edff700 1 Processor -- start 2023-11-28T11:19:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.862+0000 7ff21edff700 1 -- start start 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21edff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff218161340 0x7ff2180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21edff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180d1950 con 0x7ff21815c9b0 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21edff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180b5520 con 0x7ff2180d2320 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21edff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180b5800 con 0x7ff218161340 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21cb9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff21815c9b0 0x7ff2180b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53070/0 (socket says 172.21.15.53:53070) 2023-11-28T11:19:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff21cb9b700 1 -- 172.21.15.53:0/3260271150 learned_addr learned my addr 172.21.15.53:0/3260271150 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff217fff700 1 --2- 172.21.15.53:0/3260271150 >> v2:172.21.15.157:3300/0 conn(0x7ff218161340 0x7ff2180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904330111 0 0) 0x7ff2180d1950 con 0x7ff21815c9b0 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.863+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2080035b0 con 0x7ff21815c9b0 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469858701 0 0) 0x7ff2080035b0 con 0x7ff21815c9b0 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 >> v2:172.21.15.157:3300/0 conn(0x7ff218161340 msgr2=0x7ff2180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 --2- 172.21.15.53:0/3260271150 >> v2:172.21.15.157:3300/0 conn(0x7ff218161340 0x7ff2180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 >> v1:172.21.15.157:6789/0 conn(0x7ff2180d2320 legacy=0x7ff2180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180d1300 con 0x7ff21815c9b0 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7ff20c001670 con 0x7ff21815c9b0 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 >> v1:172.21.15.53:6789/0 conn(0x7ff21815c9b0 legacy=0x7ff2180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff208003a00 0x7ff208003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208004850 0x7ff208004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 0x7ff20800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180b5800 con 0x7ff208005200 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20800c110 con 0x7ff208004850 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20800c2d0 con 0x7ff208003a00 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff21d39c700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 0x7ff20800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff21cb9b700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff208003a00 0x7ff208003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.864+0000 7ff217fff700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208004850 0x7ff208004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 -- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff208003a00 msgr2=0x7ff208003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff208003a00 0x7ff208003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 -- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208004850 msgr2=0x7ff208004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208004850 0x7ff208004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 -- 172.21.15.53:0/3260271150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080035b0 con 0x7ff208005200 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 -- 172.21.15.53:0/3260271150 reap_dead start 2023-11-28T11:19:28.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff21d39c700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 0x7ff20800ba10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c160 tx=0x7ff208005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fca01e5cf5f09cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210001db0 con 0x7ff208005200 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000ccc0 con 0x7ff208005200 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.865+0000 7ff2177fe700 1 -- 172.21.15.53:0/3260271150 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210003c40 con 0x7ff208005200 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 -- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 msgr2=0x7ff20800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 0x7ff20800ba10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c160 tx=0x7ff208005880 comp rx=0 tx=0).stop 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 -- 172.21.15.53:0/3260271150 shutdown_connections 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 --2- 172.21.15.53:0/3260271150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff208005200 0x7ff20800ba10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 -- 172.21.15.53:0/3260271150 >> 172.21.15.53:0/3260271150 conn(0x7ff218072600 msgr2=0x7ff2180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:28.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 -- 172.21.15.53:0/3260271150 shutdown_connections 2023-11-28T11:19:28.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.866+0000 7ff21edff700 1 -- 172.21.15.53:0/3260271150 wait complete. 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.868+0000 7ff21edff700 1 Processor -- start 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.868+0000 7ff21edff700 1 -- start start 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 0x7ff2180cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 0x7ff2180c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815d550 con 0x7ff2180c5df0 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815d830 con 0x7ff208005200 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815db10 con 0x7ff21815e8c0 2023-11-28T11:19:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21d39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21d39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39694/0 (socket says 172.21.15.53:39694) 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21d39c700 1 -- 172.21.15.53:0/440856010 learned_addr learned my addr 172.21.15.53:0/440856010 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff217fff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 0x7ff2180c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.870+0000 7ff21cb9b700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 0x7ff2180cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 -- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 msgr2=0x7ff2180c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 0x7ff2180c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 -- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 msgr2=0x7ff2180cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 0x7ff2180cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 -- 172.21.15.53:0/440856010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21815cd80 con 0x7ff2180c5df0 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21cb9b700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 0x7ff2180cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff217fff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 0x7ff2180c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21d39c700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff210005040 tx=0x7ff2100014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210005070 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2100051d0 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2180d15e0 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210005560 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.871+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21815cf10 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.872+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2180745c0 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.873+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff210010050 con 0x7ff2180c5df0 2023-11-28T11:19:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.874+0000 7ff2157fa700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 0x7ff1fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.875+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff210086ee0 con 0x7ff2180c5df0 2023-11-28T11:19:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.875+0000 7ff21cb9b700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 0x7ff1fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.876+0000 7ff21cb9b700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 0x7ff1fc06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:28.878+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff210053740 con 0x7ff2180c5df0 2023-11-28T11:19:29.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.081+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2180d08a0 con 0x7ff2180c5df0 2023-11-28T11:19:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.083+0000 7ff2157fa700 1 -- 172.21.15.53:0/440856010 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17979 (secure 0 0 0) 0x7ff210016030 con 0x7ff2180c5df0 2023-11-28T11:19:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:29.085 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:00.492727+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 msgr2=0x7ff1fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 0x7ff1fc06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c009040 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 msgr2=0x7ff2180c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff210005040 tx=0x7ff2100014a0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 shutdown_connections 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1fc0682f0 0x7ff1fc06a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2180c5df0 0x7ff2180c6260 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff208005200 0x7ff2180cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 --2- 172.21.15.53:0/440856010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff21815e8c0 0x7ff2180c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 >> 172.21.15.53:0/440856010 conn(0x7ff218072600 msgr2=0x7ff21815a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 shutdown_connections 2023-11-28T11:19:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.085+0000 7ff21edff700 1 -- 172.21.15.53:0/440856010 wait complete. 2023-11-28T11:19:29.105 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:19:29.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.306+0000 7f52b71ff700 1 Processor -- start 2023-11-28T11:19:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.306+0000 7f52b71ff700 1 -- start start 2023-11-28T11:19:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.307+0000 7f52b71ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.307+0000 7f52b71ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52b0161340 0x7f52b00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.307+0000 7f52b71ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b00d1950 con 0x7f52b00d2320 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.307+0000 7f52b71ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b00b5800 con 0x7f52b0161340 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.307+0000 7f52b71ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00b5520 con 0x7f52b015c9b0 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52b0161340 0x7f52b00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52b0161340 0x7f52b00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39702/0 (socket says 172.21.15.53:39702) 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 -- 172.21.15.53:0/2005620921 learned_addr learned my addr 172.21.15.53:0/2005620921 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52b4f9b700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 -- 172.21.15.53:0/2005620921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 msgr2=0x7f52b00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 -- 172.21.15.53:0/2005620921 >> v1:172.21.15.157:6789/0 conn(0x7f52b00d2320 legacy=0x7f52b00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 -- 172.21.15.53:0/2005620921 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00d1300 con 0x7f52b0161340 2023-11-28T11:19:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52af7fe700 1 -- 172.21.15.53:0/2005620921 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250270758 0 0) 0x7f52b00d1950 con 0x7f52b00d2320 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.308+0000 7f52affff700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00d1de0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098f0 tx=0x7f529c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3b16199b661c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52af7fe700 1 -- 172.21.15.53:0/2005620921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c011830 con 0x7f52b0161340 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52af7fe700 1 -- 172.21.15.53:0/2005620921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c011990 con 0x7f52b0161340 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52af7fe700 1 -- 172.21.15.53:0/2005620921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c011d20 con 0x7f52b0161340 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 -- 172.21.15.53:0/2005620921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 msgr2=0x7f52b00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00d1de0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098f0 tx=0x7f529c009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 -- 172.21.15.53:0/2005620921 shutdown_connections 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 --2- 172.21.15.53:0/2005620921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00d1de0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 -- 172.21.15.53:0/2005620921 >> 172.21.15.53:0/2005620921 conn(0x7f52b0072600 msgr2=0x7f52b00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 -- 172.21.15.53:0/2005620921 shutdown_connections 2023-11-28T11:19:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.309+0000 7f52b71ff700 1 -- 172.21.15.53:0/2005620921 wait complete. 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.311+0000 7f52b71ff700 1 Processor -- start 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.311+0000 7f52b71ff700 1 -- start start 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 0x7f52b01e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0140e10 con 0x7f52b0161340 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01e8430 con 0x7f52b00d2320 2023-11-28T11:19:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b71ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01e6460 con 0x7f52b015c9b0 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39714/0 (socket says 172.21.15.53:39714) 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.312+0000 7f52b579c700 1 -- 172.21.15.53:0/1936269494 learned_addr learned my addr 172.21.15.53:0/1936269494 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b4f9b700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 0x7f52b01e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52affff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 -- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 msgr2=0x7f52b00b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 -- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 msgr2=0x7f52b01e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 0x7f52b01e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 -- 172.21.15.53:0/1936269494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c009550 con 0x7f52b0161340 2023-11-28T11:19:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b4f9b700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 0x7f52b01e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52affff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b579c700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c7d0 tx=0x7f52a800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8003e30 con 0x7f52b0161340 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a8005070 con 0x7f52b0161340 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005420 con 0x7f52b0161340 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b00b5130 con 0x7f52b0161340 2023-11-28T11:19:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.313+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b00d15e0 con 0x7f52b0161340 2023-11-28T11:19:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.314+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b00b52c0 con 0x7f52b0161340 2023-11-28T11:19:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.315+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52a8019030 con 0x7f52b0161340 2023-11-28T11:19:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.316+0000 7f52ad7fa700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 0x7f529406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.317+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52a80853e0 con 0x7f52b0161340 2023-11-28T11:19:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.317+0000 7f52b4f9b700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 0x7f529406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.318+0000 7f52b4f9b700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 0x7f529406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009b60 tx=0x7f52a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.320+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52a8051ac0 con 0x7f52b0161340 2023-11-28T11:19:29.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.564+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52b00b9150 con 0x7f52b0161340 2023-11-28T11:19:29.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.565+0000 7f52ad7fa700 1 -- 172.21.15.53:0/1936269494 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f52a8017070 con 0x7f52b0161340 2023-11-28T11:19:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:29.568 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.567+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 msgr2=0x7f529406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.567+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 0x7f529406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009b60 tx=0x7f52a4008040 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.567+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 msgr2=0x7f52b00b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.567+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c7d0 tx=0x7f52a800cb90 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.567+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 shutdown_connections 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52940682f0 0x7f529406a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b0161340 0x7f52b00b77b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b00d2320 0x7f52b01e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 --2- 172.21.15.53:0/1936269494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b015c9b0 0x7f52b00b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 >> 172.21.15.53:0/1936269494 conn(0x7f52b0072600 msgr2=0x7f52b0160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:29.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 shutdown_connections 2023-11-28T11:19:29.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.568+0000 7f52b71ff700 1 -- 172.21.15.53:0/1936269494 wait complete. 2023-11-28T11:19:29.571 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:29.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:29.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.787+0000 7f63c2dff700 1 Processor -- start 2023-11-28T11:19:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.788+0000 7f63c2dff700 1 -- start start 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c2dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c2dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0d1950 con 0x7f63bc0cd7c0 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c2dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0d24f0 con 0x7f63bc161340 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c2dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b5800 con 0x7f63bc15c9b0 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c139c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f63bc0cd7c0 0x7f63bc0b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53072/0 (socket says 172.21.15.53:53072) 2023-11-28T11:19:29.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c139c700 1 -- 172.21.15.53:0/1959469523 learned_addr learned my addr 172.21.15.53:0/1959469523 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.789+0000 7f63c0b9b700 1 --2- 172.21.15.53:0/1959469523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878080992 0 0) 0x7f63bc0d1950 con 0x7f63bc0cd7c0 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac0035b0 con 0x7f63bc0cd7c0 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062956906 0 0) 0x7f63bc0d24f0 con 0x7f63bc161340 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63bc0d1950 con 0x7f63bc161340 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63c0b9b700 1 -- 172.21.15.53:0/1959469523 >> v1:172.21.15.157:6789/0 conn(0x7f63bc161340 legacy=0x7f63bc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63c0b9b700 1 -- 172.21.15.53:0/1959469523 >> v1:172.21.15.53:6789/0 conn(0x7f63bc0cd7c0 legacy=0x7f63bc0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63c0b9b700 1 -- 172.21.15.53:0/1959469523 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0b5520 con 0x7f63bc15c9b0 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.790+0000 7f63c0b9b700 1 --2- 172.21.15.53:0/1959469523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc0b4bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009870 tx=0x7f63b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0f73ea22cc590d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0011830 con 0x7f63bc15c9b0 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b0011990 con 0x7f63bc15c9b0 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63bb7fe700 1 -- 172.21.15.53:0/1959469523 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0011d20 con 0x7f63bc15c9b0 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 -- 172.21.15.53:0/1959469523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 msgr2=0x7f63bc0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 --2- 172.21.15.53:0/1959469523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc0b4bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009870 tx=0x7f63b0009b80 comp rx=0 tx=0).stop 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 -- 172.21.15.53:0/1959469523 shutdown_connections 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 --2- 172.21.15.53:0/1959469523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc0b4bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 -- 172.21.15.53:0/1959469523 >> 172.21.15.53:0/1959469523 conn(0x7f63bc072600 msgr2=0x7f63bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 -- 172.21.15.53:0/1959469523 shutdown_connections 2023-11-28T11:19:29.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.791+0000 7f63c2dff700 1 -- 172.21.15.53:0/1959469523 wait complete. 2023-11-28T11:19:29.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.793+0000 7f63c2dff700 1 Processor -- start 2023-11-28T11:19:29.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.793+0000 7f63c2dff700 1 -- start start 2023-11-28T11:19:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0cd7c0 0x7f63bc1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cd2a0 con 0x7f63bc161340 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0ccad0 con 0x7f63bc0cd7c0 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c2dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc14aac0 con 0x7f63bc15c9b0 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c139c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c139c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39716/0 (socket says 172.21.15.53:39716) 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.794+0000 7f63c139c700 1 -- 172.21.15.53:0/3750692786 learned_addr learned my addr 172.21.15.53:0/3750692786 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c0b9b700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0cd7c0 0x7f63bc1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63bbfff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 -- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 msgr2=0x7f63bc14a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc14a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 -- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0cd7c0 msgr2=0x7f63bc1e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0cd7c0 0x7f63bc1e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 -- 172.21.15.53:0/3750692786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00094a0 con 0x7f63bc161340 2023-11-28T11:19:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c139c700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63b400caf0 tx=0x7f63b400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b400aaa0 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b400ac00 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4012400 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.795+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc0d15e0 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.796+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc0d1300 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.796+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0b5130 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.797+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f63b4019030 con 0x7f63bc161340 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.799+0000 7f63b97fa700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 0x7f63a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:29.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.800+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f63b40858a0 con 0x7f63bc161340 2023-11-28T11:19:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.800+0000 7f63c0b9b700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 0x7f63a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.801+0000 7f63c0b9b700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 0x7f63a006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:29.803+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f63b4052010 con 0x7f63bc161340 2023-11-28T11:19:30.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.052+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63bc0745c0 con 0x7f63bc161340 2023-11-28T11:19:30.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.052+0000 7f63b97fa700 1 -- 172.21.15.53:0/3750692786 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f63b4055630 con 0x7f63bc161340 2023-11-28T11:19:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:30.054 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:18:59.490713+0000","modified":"2023-11-28T11:19:02.510023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54722},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54722":{"gid":54722,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3764823575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3764823575},{"type":"v1","addr":"172.21.15.53:6834","nonce":3764823575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:19:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 msgr2=0x7f63a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 0x7f63a006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c040 comp rx=0 tx=0).stop 2023-11-28T11:19:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 msgr2=0x7f63bc1e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63b400caf0 tx=0x7f63b400ce00 comp rx=0 tx=0).stop 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 shutdown_connections 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f63a00682f0 0x7f63a006a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63bc161340 0x7f63bc1e6440 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0cd7c0 0x7f63bc1e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 --2- 172.21.15.53:0/3750692786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15c9b0 0x7f63bc14a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 >> 172.21.15.53:0/3750692786 conn(0x7f63bc072600 msgr2=0x7f63bc073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 shutdown_connections 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.055+0000 7f63c2dff700 1 -- 172.21.15.53:0/3750692786 wait complete. 2023-11-28T11:19:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 16 2023-11-28T11:19:30.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:30.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.274+0000 7f6a0f1ff700 1 Processor -- start 2023-11-28T11:19:30.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.275+0000 7f6a0f1ff700 1 -- start start 2023-11-28T11:19:30.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a08161340 0x7f6a080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a080d2320 0x7f6a080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0f1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a080d1950 con 0x7f6a0815c9b0 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080b5800 con 0x7f6a080d2320 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080b5300 con 0x7f6a08161340 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a080d2320 0x7f6a080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a080d2320 0x7f6a080cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39732/0 (socket says 172.21.15.53:39732) 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 -- 172.21.15.53:0/3745210958 learned_addr learned my addr 172.21.15.53:0/3745210958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a07fff700 1 --2- 172.21.15.53:0/3745210958 >> v2:172.21.15.157:3300/0 conn(0x7f6a08161340 0x7f6a080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 -- 172.21.15.53:0/3745210958 >> v2:172.21.15.157:3300/0 conn(0x7f6a08161340 msgr2=0x7f6a080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 --2- 172.21.15.53:0/3745210958 >> v2:172.21.15.157:3300/0 conn(0x7f6a08161340 0x7f6a080d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.276+0000 7f6a0d79c700 1 -- 172.21.15.53:0/3745210958 >> v1:172.21.15.157:6789/0 conn(0x7f6a0815c9b0 legacy=0x7f6a080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a0d79c700 1 -- 172.21.15.53:0/3745210958 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080b5520 con 0x7f6a080d2320 2023-11-28T11:19:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a0d79c700 1 --2- 172.21.15.53:0/3745210958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080cd7c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6a080685c0 tx=0x7f6a0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6120ca4cdb014b7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a077fe700 1 -- 172.21.15.53:0/3745210958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000e070 con 0x7f6a080d2320 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a077fe700 1 -- 172.21.15.53:0/3745210958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0000b980 con 0x7f6a080d2320 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a077fe700 1 -- 172.21.15.53:0/3745210958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000bd30 con 0x7f6a080d2320 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3745210958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 msgr2=0x7f6a080cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.277+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3745210958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080cd7c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6a080685c0 tx=0x7f6a0000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3745210958 shutdown_connections 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3745210958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080cd7c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3745210958 >> v2:172.21.15.157:3300/0 conn(0x7f6a08161340 0x7f6a080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3745210958 >> 172.21.15.53:0/3745210958 conn(0x7f6a08072600 msgr2=0x7f6a080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3745210958 shutdown_connections 2023-11-28T11:19:30.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.278+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3745210958 wait complete. 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.279+0000 7f6a0f1ff700 1 Processor -- start 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 -- start start 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a0815c9b0 0x7f6a0814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a08161340 0x7f6a08157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081e64c0 con 0x7f6a080d2320 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081e67a0 con 0x7f6a08161340 2023-11-28T11:19:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.280+0000 7f6a0f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081e6a80 con 0x7f6a0815c9b0 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39744/0 (socket says 172.21.15.53:39744) 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 -- 172.21.15.53:0/3356767086 learned_addr learned my addr 172.21.15.53:0/3356767086 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0d79c700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a08161340 0x7f6a08157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a07fff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a0815c9b0 0x7f6a0814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 -- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a0815c9b0 msgr2=0x7f6a0814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a0815c9b0 0x7f6a0814dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 -- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a08161340 msgr2=0x7f6a08157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a08161340 0x7f6a08157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 -- 172.21.15.53:0/3356767086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00009b30 con 0x7f6a080d2320 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.281+0000 7f6a0cf9b700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c430 tx=0x7f69fc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.282+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0052d0 con 0x7f6a080d2320 2023-11-28T11:19:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.282+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc005450 con 0x7f6a080d2320 2023-11-28T11:19:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.282+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a080d15e0 con 0x7f6a080d2320 2023-11-28T11:19:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.282+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc006aa0 con 0x7f6a080d2320 2023-11-28T11:19:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.282+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a080d1300 con 0x7f6a080d2320 2023-11-28T11:19:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.283+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0815cd80 con 0x7f6a080d2320 2023-11-28T11:19:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.284+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f69fc010050 con 0x7f6a080d2320 2023-11-28T11:19:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.286+0000 7f6a057fa700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 0x7f69ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.286+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69fc084e70 con 0x7f6a080d2320 2023-11-28T11:19:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.286+0000 7f6a07fff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 0x7f69ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.287+0000 7f6a07fff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 0x7f69ec06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a080c69b0 tx=0x7f69f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.289+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f69fc04a5e0 con 0x7f6a080d2320 2023-11-28T11:19:30.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.492+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a080745c0 con 0x7f6a080d2320 2023-11-28T11:19:30.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.494+0000 7f6a057fa700 1 -- 172.21.15.53:0/3356767086 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f69fc0515e0 con 0x7f6a080d2320 2023-11-28T11:19:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:30.495 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:00.492727+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:18:57.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:18:58.463907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 msgr2=0x7f69ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 0x7f69ec06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a080c69b0 tx=0x7f69f4008040 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 msgr2=0x7f6a080c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c430 tx=0x7f69fc00c7f0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 shutdown_connections 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69ec0682f0 0x7f69ec06a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a08161340 0x7f6a08157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a080d2320 0x7f6a080c8e40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 --2- 172.21.15.53:0/3356767086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a0815c9b0 0x7f6a0814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 >> 172.21.15.53:0/3356767086 conn(0x7f6a08072600 msgr2=0x7f6a080b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 shutdown_connections 2023-11-28T11:19:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.496+0000 7f6a0f1ff700 1 -- 172.21.15.53:0/3356767086 wait complete. 2023-11-28T11:19:30.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:19:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.718+0000 7f6cdd1ff700 1 Processor -- start 2023-11-28T11:19:30.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.718+0000 7f6cdd1ff700 1 -- start start 2023-11-28T11:19:30.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.719+0000 7f6cdd1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cd815c9b0 0x7f6cd80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.719+0000 7f6cdd1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 0x7f6cd80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.719+0000 7f6cdd1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd80d1950 con 0x7f6cd80d2320 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cdd1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd80b5800 con 0x7f6cd815c9b0 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cdd1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80b5520 con 0x7f6cd8161340 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd759e700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6cd80d2320 0x7f6cd80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53086/0 (socket says 172.21.15.53:53086) 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd759e700 1 -- 172.21.15.53:0/4160504150 learned_addr learned my addr 172.21.15.53:0/4160504150 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd6d9d700 1 --2- 172.21.15.53:0/4160504150 >> v2:172.21.15.157:3300/0 conn(0x7f6cd815c9b0 0x7f6cd80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd659c700 1 --2- 172.21.15.53:0/4160504150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 0x7f6cd80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd5d9b700 1 -- 172.21.15.53:0/4160504150 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176454043 0 0) 0x7f6cd80d1950 con 0x7f6cd80d2320 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.720+0000 7f6cd5d9b700 1 -- 172.21.15.53:0/4160504150 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc00035b0 con 0x7f6cd80d2320 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd659c700 1 -- 172.21.15.53:0/4160504150 >> v2:172.21.15.157:3300/0 conn(0x7f6cd815c9b0 msgr2=0x7f6cd80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd659c700 1 --2- 172.21.15.53:0/4160504150 >> v2:172.21.15.157:3300/0 conn(0x7f6cd815c9b0 0x7f6cd80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd659c700 1 -- 172.21.15.53:0/4160504150 >> v1:172.21.15.53:6789/0 conn(0x7f6cd80d2320 legacy=0x7f6cd80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd659c700 1 -- 172.21.15.53:0/4160504150 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80d1300 con 0x7f6cd8161340 2023-11-28T11:19:30.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd659c700 1 --2- 172.21.15.53:0/4160504150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 0x7f6cd80d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009840 tx=0x7f6ccc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea8097ed9702367 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd5d9b700 1 -- 172.21.15.53:0/4160504150 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011830 con 0x7f6cd8161340 2023-11-28T11:19:30.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd5d9b700 1 -- 172.21.15.53:0/4160504150 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc011990 con 0x7f6cd8161340 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.721+0000 7f6cd5d9b700 1 -- 172.21.15.53:0/4160504150 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011d20 con 0x7f6cd8161340 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/4160504150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 msgr2=0x7f6cd80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/4160504150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 0x7f6cd80d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009840 tx=0x7f6ccc009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/4160504150 shutdown_connections 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/4160504150 >> v2:172.21.15.157:3300/0 conn(0x7f6cd815c9b0 0x7f6cd80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/4160504150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd8161340 0x7f6cd80d1de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/4160504150 >> 172.21.15.53:0/4160504150 conn(0x7f6cd8072600 msgr2=0x7f6cd80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/4160504150 shutdown_connections 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.722+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/4160504150 wait complete. 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.724+0000 7f6cdd1ff700 1 Processor -- start 2023-11-28T11:19:30.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.724+0000 7f6cdd1ff700 1 -- start start 2023-11-28T11:19:30.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd80d2320 0x7f6cd8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd815c9b0 0x7f6cd81578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8150610 con 0x7f6cd8161340 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8157df0 con 0x7f6cd80d2320 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cdd1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81f5f20 con 0x7f6cd815c9b0 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39754/0 (socket says 172.21.15.53:39754) 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 -- 172.21.15.53:0/3519862269 learned_addr learned my addr 172.21.15.53:0/3519862269 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd6d9d700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd80d2320 0x7f6cd8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd659c700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd815c9b0 0x7f6cd81578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 -- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd815c9b0 msgr2=0x7f6cd81578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd815c9b0 0x7f6cd81578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 -- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd80d2320 msgr2=0x7f6cd8150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd80d2320 0x7f6cd8150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.725+0000 7f6cd759e700 1 -- 172.21.15.53:0/3519862269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0094a0 con 0x7f6cd8161340 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cd759e700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000caa0 tx=0x7f6cd000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000a9e0 con 0x7f6cd8161340 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd000ab40 con 0x7f6cd8161340 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd80b5130 con 0x7f6cd8161340 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0012400 con 0x7f6cd8161340 2023-11-28T11:19:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.726+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd80d15e0 con 0x7f6cd8161340 2023-11-28T11:19:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.727+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd80b52c0 con 0x7f6cd8161340 2023-11-28T11:19:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.728+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6cd0019030 con 0x7f6cd8161340 2023-11-28T11:19:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.729+0000 7f6cbf7fe700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 0x7f6cc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.729+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cd0085900 con 0x7f6cd8161340 2023-11-28T11:19:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.729+0000 7f6cd6d9d700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 0x7f6cc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:30.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.730+0000 7f6cd6d9d700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 0x7f6cc406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009be0 tx=0x7f6cc8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.733+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6cd00520b0 con 0x7f6cd8161340 2023-11-28T11:19:30.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:30.984+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6cd80745c0 con 0x7f6cd8161340 2023-11-28T11:19:31.829 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-11-28T11:19:31.828+0000 7fa6ddbf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:19:31.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.837+0000 7f6cbf7fe700 1 -- 172.21.15.53:0/3519862269 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6cd00556d0 con 0x7f6cd8161340 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 msgr2=0x7f6cc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 0x7f6cc406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009be0 tx=0x7f6cc8009210 comp rx=0 tx=0).stop 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 msgr2=0x7f6cd81f5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000caa0 tx=0x7f6cd000cdb0 comp rx=0 tx=0).stop 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 shutdown_connections 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6cc40682f0 0x7f6cc406a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8161340 0x7f6cd81f5550 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd80d2320 0x7f6cd8150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 --2- 172.21.15.53:0/3519862269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd815c9b0 0x7f6cd81578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 >> 172.21.15.53:0/3519862269 conn(0x7f6cd8072600 msgr2=0x7f6cd8160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 shutdown_connections 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:31.840+0000 7f6cdd1ff700 1 -- 172.21.15.53:0/3519862269 wait complete. 2023-11-28T11:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:19:31.851 INFO:tasks.ceph.mds.a.smithi053.stderr: -7211> 2023-11-28T11:19:00.938+0000 7f5c9b3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:19:31.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:19:31.910 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:19:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.072+0000 7f794dfff700 1 Processor -- start 2023-11-28T11:19:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.072+0000 7f794dfff700 1 -- start start 2023-11-28T11:19:32.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f794dfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 0x7f79480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f794dfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79480cd7c0 0x7f79480cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f794dfff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79480d1950 con 0x7f7948161340 2023-11-28T11:19:32.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f794dfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480b5800 con 0x7f79480cd7c0 2023-11-28T11:19:32.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f794dfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480ce140 con 0x7f794815c9b0 2023-11-28T11:19:32.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f7947fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79480cd7c0 0x7f79480cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f7947fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79480cd7c0 0x7f79480cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55662/0 (socket says 172.21.15.53:55662) 2023-11-28T11:19:32.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f7947fff700 1 -- 172.21.15.53:0/1584134087 learned_addr learned my addr 172.21.15.53:0/1584134087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:32.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.073+0000 7f79477fe700 1 --2- 172.21.15.53:0/1584134087 >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 0x7f79480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7947fff700 1 -- 172.21.15.53:0/1584134087 >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 msgr2=0x7f79480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:32.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7947fff700 1 --2- 172.21.15.53:0/1584134087 >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 0x7f79480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7947fff700 1 -- 172.21.15.53:0/1584134087 >> v1:172.21.15.157:6789/0 conn(0x7f7948161340 legacy=0x7f79480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7947fff700 1 -- 172.21.15.53:0/1584134087 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480d24f0 con 0x7f79480cd7c0 2023-11-28T11:19:32.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f79477fe700 1 --2- 172.21.15.53:0/1584134087 >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 0x7f79480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:32.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7947fff700 1 --2- 172.21.15.53:0/1584134087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79480cd7c0 0x7f79480cdc00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7948067460 tx=0x7f794000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed311b66b13b2490 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:32.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7946ffd700 1 -- 172.21.15.53:0/1584134087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794000e070 con 0x7f79480cd7c0 2023-11-28T11:19:32.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7946ffd700 1 -- 172.21.15.53:0/1584134087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794000b9c0 con 0x7f79480cd7c0 2023-11-28T11:19:32.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.074+0000 7f7946ffd700 1 -- 172.21.15.53:0/1584134087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794000bd70 con 0x7f79480cd7c0 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 -- 172.21.15.53:0/1584134087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79480cd7c0 msgr2=0x7f79480cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 --2- 172.21.15.53:0/1584134087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79480cd7c0 0x7f79480cdc00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7948067460 tx=0x7f794000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 -- 172.21.15.53:0/1584134087 shutdown_connections 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 --2- 172.21.15.53:0/1584134087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79480cd7c0 0x7f79480cdc00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 --2- 172.21.15.53:0/1584134087 >> v2:172.21.15.157:3300/0 conn(0x7f794815c9b0 0x7f79480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 -- 172.21.15.53:0/1584134087 >> 172.21.15.53:0/1584134087 conn(0x7f7948072600 msgr2=0x7f79480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 -- 172.21.15.53:0/1584134087 shutdown_connections 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.075+0000 7f794dfff700 1 -- 172.21.15.53:0/1584134087 wait complete. 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.077+0000 7f794dfff700 1 Processor -- start 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.077+0000 7f794dfff700 1 -- start start 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79480cd7c0 0x7f79480cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7948161340 0x7f79481f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79480cc7b0 con 0x7f794815c9b0 2023-11-28T11:19:32.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79480cb920 con 0x7f7948161340 2023-11-28T11:19:32.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f794dfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79481e7ba0 con 0x7f79480cd7c0 2023-11-28T11:19:32.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55666/0 (socket says 172.21.15.53:55666) 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 -- 172.21.15.53:0/3022200775 learned_addr learned my addr 172.21.15.53:0/3022200775 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f7947fff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7948161340 0x7f79481f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f79477fe700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79480cd7c0 0x7f79480cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 -- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79480cd7c0 msgr2=0x7f79480cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79480cd7c0 0x7f79480cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 -- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7948161340 msgr2=0x7f79481f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7948161340 0x7f79481f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.078+0000 7f793ffff700 1 -- 172.21.15.53:0/3022200775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940009b30 con 0x7f794815c9b0 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f793ffff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f793000c450 tx=0x7f793000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300052d0 con 0x7f794815c9b0 2023-11-28T11:19:32.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930005430 con 0x7f794815c9b0 2023-11-28T11:19:32.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300038c0 con 0x7f794815c9b0 2023-11-28T11:19:32.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79480d1300 con 0x7f794815c9b0 2023-11-28T11:19:32.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.079+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79480b5520 con 0x7f794815c9b0 2023-11-28T11:19:32.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.080+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79480d15e0 con 0x7f794815c9b0 2023-11-28T11:19:32.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.081+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7930019030 con 0x7f794815c9b0 2023-11-28T11:19:32.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.082+0000 7f7944ff9700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 0x7f792806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:32.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.082+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7930085060 con 0x7f794815c9b0 2023-11-28T11:19:32.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.082+0000 7f79477fe700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 0x7f792806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:32.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.083+0000 7f79477fe700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 0x7f792806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f79380098f0 tx=0x7f7938008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:32.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.086+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7930051770 con 0x7f794815c9b0 2023-11-28T11:19:32.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.337+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f79480cc8f0 con 0x7f794815c9b0 2023-11-28T11:19:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.837+0000 7f7944ff9700 1 -- 172.21.15.53:0/3022200775 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f79480cc8f0 con 0x7f794815c9b0 2023-11-28T11:19:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 msgr2=0x7f792806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 0x7f792806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f79380098f0 tx=0x7f7938008040 comp rx=0 tx=0).stop 2023-11-28T11:19:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 msgr2=0x7f79481e7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f793000c450 tx=0x7f793000c760 comp rx=0 tx=0).stop 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 shutdown_connections 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f79280682f0 0x7f792806a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79480cd7c0 0x7f79480cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7948161340 0x7f79481f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 --2- 172.21.15.53:0/3022200775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f794815c9b0 0x7f79481e7660 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 >> 172.21.15.53:0/3022200775 conn(0x7f7948072600 msgr2=0x7f79480b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 shutdown_connections 2023-11-28T11:19:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:32.840+0000 7f794dfff700 1 -- 172.21.15.53:0/3022200775 wait complete. 2023-11-28T11:19:32.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:19:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.108+0000 7f114e1ff700 1 Processor -- start 2023-11-28T11:19:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.108+0000 7f114e1ff700 1 -- start start 2023-11-28T11:19:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f114e1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f114815a760 0x7f1148161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f114e1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114815e0a0 con 0x7f11480cded0 2023-11-28T11:19:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f114e1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1148160d70 con 0x7f114815dc30 2023-11-28T11:19:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f114e1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114815ab30 con 0x7f114815a760 2023-11-28T11:19:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f11477fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f114815a760 0x7f1148161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f11477fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f114815a760 0x7f1148161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55680/0 (socket says 172.21.15.53:55680) 2023-11-28T11:19:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f11477fe700 1 -- 172.21.15.53:0/1085123846 learned_addr learned my addr 172.21.15.53:0/1085123846 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:33.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.109+0000 7f1147fff700 1 --1- 172.21.15.53:0/1085123846 >> v1:172.21.15.53:6789/0 conn(0x7f11480cded0 0x7f11480d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48346/0 (socket says 172.21.15.53:48346) 2023-11-28T11:19:33.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f11477fe700 1 -- 172.21.15.53:0/1085123846 >> v1:172.21.15.157:6789/0 conn(0x7f114815dc30 legacy=0x7f11480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f11477fe700 1 -- 172.21.15.53:0/1085123846 >> v1:172.21.15.53:6789/0 conn(0x7f11480cded0 legacy=0x7f11480d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f11477fe700 1 -- 172.21.15.53:0/1085123846 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480b5390 con 0x7f114815a760 2023-11-28T11:19:33.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f1146ffd700 1 -- 172.21.15.53:0/1085123846 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218448147 0 0) 0x7f114815e0a0 con 0x7f11480cded0 2023-11-28T11:19:33.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f11477fe700 1 --2- 172.21.15.53:0/1085123846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f1148161200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11380098f0 tx=0x7f1138009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eacb2257d167985 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:33.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f1146ffd700 1 -- 172.21.15.53:0/1085123846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800b8a0 con 0x7f114815a760 2023-11-28T11:19:33.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f1146ffd700 1 -- 172.21.15.53:0/1085123846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800ba00 con 0x7f114815a760 2023-11-28T11:19:33.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.110+0000 7f1146ffd700 1 -- 172.21.15.53:0/1085123846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800bd90 con 0x7f114815a760 2023-11-28T11:19:33.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 -- 172.21.15.53:0/1085123846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 msgr2=0x7f1148161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1085123846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f1148161200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11380098f0 tx=0x7f1138009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 -- 172.21.15.53:0/1085123846 shutdown_connections 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1085123846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f1148161200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 -- 172.21.15.53:0/1085123846 >> 172.21.15.53:0/1085123846 conn(0x7f1148072620 msgr2=0x7f11480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 -- 172.21.15.53:0/1085123846 shutdown_connections 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.111+0000 7f114e1ff700 1 -- 172.21.15.53:0/1085123846 wait complete. 2023-11-28T11:19:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.113+0000 7f114e1ff700 1 Processor -- start 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.113+0000 7f114e1ff700 1 -- start start 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 0x7f11481587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f114815dc30 0x7f11480cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11480b7640 con 0x7f114815a760 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11481592b0 con 0x7f11480cded0 2023-11-28T11:19:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f114e1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1148158d30 con 0x7f114815dc30 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55694/0 (socket says 172.21.15.53:55694) 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 -- 172.21.15.53:0/1074422604 learned_addr learned my addr 172.21.15.53:0/1074422604 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f1147fff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f114815dc30 0x7f11480cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f11477fe700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 0x7f11481587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 -- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f114815dc30 msgr2=0x7f11480cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f114815dc30 0x7f11480cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 -- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 msgr2=0x7f11481587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 0x7f11481587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f113ffff700 1 -- 172.21.15.53:0/1074422604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009550 con 0x7f114815a760 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.114+0000 7f11477fe700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 0x7f11481587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f113ffff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f113000c400 tx=0x7f113000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300052d0 con 0x7f114815a760 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1130005430 con 0x7f114815a760 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148160720 con 0x7f114815a760 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300038a0 con 0x7f114815a760 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.115+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11480b5670 con 0x7f114815a760 2023-11-28T11:19:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.116+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1148160a00 con 0x7f114815a760 2023-11-28T11:19:33.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.117+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1130019030 con 0x7f114815a760 2023-11-28T11:19:33.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.118+0000 7f1144ff9700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 0x7f112806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:33.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.118+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1130085090 con 0x7f114815a760 2023-11-28T11:19:33.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.118+0000 7f11477fe700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 0x7f112806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:33.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.119+0000 7f11477fe700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 0x7f112806a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1138009130 tx=0x7f1138008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:33.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.122+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1130051740 con 0x7f114815a760 2023-11-28T11:19:33.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.325+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f114806b990 con 0x7f114815a760 2023-11-28T11:19:33.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.840+0000 7f1144ff9700 1 -- 172.21.15.53:0/1074422604 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11300170d0 con 0x7f114815a760 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 msgr2=0x7f112806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 0x7f112806a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1138009130 tx=0x7f1138008030 comp rx=0 tx=0).stop 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 msgr2=0x7f11480ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f113000c400 tx=0x7f113000c7c0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 shutdown_connections 2023-11-28T11:19:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f11280682f0 0x7f112806a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f114815dc30 0x7f11480cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11480cded0 0x7f11481587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 --2- 172.21.15.53:0/1074422604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114815a760 0x7f11480ca240 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f113000c400 tx=0x7f113000c7c0 comp rx=0 tx=0).stop 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 >> 172.21.15.53:0/1074422604 conn(0x7f1148072620 msgr2=0x7f114815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.846+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 shutdown_connections 2023-11-28T11:19:33.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:33.847+0000 7f114e1ff700 1 -- 172.21.15.53:0/1074422604 wait complete. 2023-11-28T11:19:33.849 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:19:33.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:19:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.080+0000 7fef487ff700 1 Processor -- start 2023-11-28T11:19:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.080+0000 7fef487ff700 1 -- start start 2023-11-28T11:19:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef487ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef4015a760 0x7fef40161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef487ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 0x7fef400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef487ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4015e0a0 con 0x7fef4015dc30 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef487ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4015ab30 con 0x7fef4015a760 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef487ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef40160d70 con 0x7fef400cded0 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef4659b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef4015a760 0x7fef40161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef4659b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef4015a760 0x7fef40161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55700/0 (socket says 172.21.15.53:55700) 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.081+0000 7fef4659b700 1 -- 172.21.15.53:0/335491748 learned_addr learned my addr 172.21.15.53:0/335491748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef46d9c700 1 --2- 172.21.15.53:0/335491748 >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 0x7fef400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef4659b700 1 -- 172.21.15.53:0/335491748 >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 msgr2=0x7fef400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef4659b700 1 --2- 172.21.15.53:0/335491748 >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 0x7fef400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef4659b700 1 -- 172.21.15.53:0/335491748 >> v1:172.21.15.157:6789/0 conn(0x7fef4015dc30 legacy=0x7fef400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef4659b700 1 -- 172.21.15.53:0/335491748 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400b5130 con 0x7fef4015a760 2023-11-28T11:19:34.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef46d9c700 1 --2- 172.21.15.53:0/335491748 >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 0x7fef400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef4659b700 1 --2- 172.21.15.53:0/335491748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4015a760 0x7fef40161200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef40067500 tx=0x7fef3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c062d0702d459e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef45599700 1 -- 172.21.15.53:0/335491748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c004230 con 0x7fef4015a760 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef45599700 1 -- 172.21.15.53:0/335491748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c004390 con 0x7fef4015a760 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.082+0000 7fef45599700 1 -- 172.21.15.53:0/335491748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0036a0 con 0x7fef4015a760 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 -- 172.21.15.53:0/335491748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4015a760 msgr2=0x7fef40161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 --2- 172.21.15.53:0/335491748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4015a760 0x7fef40161200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef40067500 tx=0x7fef3c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 -- 172.21.15.53:0/335491748 shutdown_connections 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 --2- 172.21.15.53:0/335491748 >> v2:172.21.15.157:3300/0 conn(0x7fef400cded0 0x7fef400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 --2- 172.21.15.53:0/335491748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4015a760 0x7fef40161200 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 -- 172.21.15.53:0/335491748 >> 172.21.15.53:0/335491748 conn(0x7fef40072620 msgr2=0x7fef400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 -- 172.21.15.53:0/335491748 shutdown_connections 2023-11-28T11:19:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.083+0000 7fef487ff700 1 -- 172.21.15.53:0/335491748 wait complete. 2023-11-28T11:19:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.085+0000 7fef487ff700 1 Processor -- start 2023-11-28T11:19:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.085+0000 7fef487ff700 1 -- start start 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef4015a760 0x7fef401e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 0x7fef401e9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400c9840 con 0x7fef400cded0 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401e8770 con 0x7fef4015dc30 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef487ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef401e8a50 con 0x7fef4015a760 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55714/0 (socket says 172.21.15.53:55714) 2023-11-28T11:19:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 -- 172.21.15.53:0/1686336614 learned_addr learned my addr 172.21.15.53:0/1686336614 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef46d9c700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 0x7fef401e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef45d9a700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef4015a760 0x7fef401e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 -- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef4015a760 msgr2=0x7fef401e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef4015a760 0x7fef401e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 -- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 msgr2=0x7fef401e9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.086+0000 7fef4659b700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 0x7fef401e9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef4659b700 1 -- 172.21.15.53:0/1686336614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c009550 con 0x7fef400cded0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef46d9c700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 0x7fef401e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef4659b700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009130 tx=0x7fef3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003dc0 con 0x7fef400cded0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c01f070 con 0x7fef400cded0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c011870 con 0x7fef400cded0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef400b5670 con 0x7fef400cded0 2023-11-28T11:19:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.087+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef400b5390 con 0x7fef400cded0 2023-11-28T11:19:34.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.088+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40160720 con 0x7fef400cded0 2023-11-28T11:19:34.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.089+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef3c017020 con 0x7fef400cded0 2023-11-28T11:19:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.090+0000 7fef36ffd700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.090+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fef3c0871a0 con 0x7fef400cded0 2023-11-28T11:19:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.090+0000 7fef45d9a700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:34.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.091+0000 7fef45d9a700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 0x7fef2406a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef40147e90 tx=0x7fef30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.094+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef3c053a70 con 0x7fef400cded0 2023-11-28T11:19:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.296+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef4004e620 con 0x7fef400cded0 2023-11-28T11:19:34.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.847+0000 7fef36ffd700 1 -- 172.21.15.53:0/1686336614 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7fef3c057090 con 0x7fef400cded0 2023-11-28T11:19:34.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 msgr2=0x7fef2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:34.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 0x7fef2406a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef40147e90 tx=0x7fef30008040 comp rx=0 tx=0).stop 2023-11-28T11:19:34.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 msgr2=0x7fef401e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:34.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009130 tx=0x7fef3c004750 comp rx=0 tx=0).stop 2023-11-28T11:19:34.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 shutdown_connections 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef4015dc30 0x7fef401e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef400cded0 0x7fef401e64e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 --2- 172.21.15.53:0/1686336614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef4015a760 0x7fef401e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 >> 172.21.15.53:0/1686336614 conn(0x7fef40072620 msgr2=0x7fef4015cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 shutdown_connections 2023-11-28T11:19:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:34.852+0000 7fef487ff700 1 -- 172.21.15.53:0/1686336614 wait complete. 2023-11-28T11:19:34.857 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-11-28T11:19:34.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:19:35.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.080+0000 7fd58fdff700 1 Processor -- start 2023-11-28T11:19:35.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.081+0000 7fd58fdff700 1 -- start start 2023-11-28T11:19:35.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd58815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd58806fb30 0x7fd5880cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 0x7fd58806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58806bf10 con 0x7fd58806fb30 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58815d730 con 0x7fd5880ce2e0 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880ce6f0 con 0x7fd588158ce0 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58d39a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd58806fb30 0x7fd5880cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58d39a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd58806fb30 0x7fd5880cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55716/0 (socket says 172.21.15.53:55716) 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58d39a700 1 -- 172.21.15.53:0/1010011654 learned_addr learned my addr 172.21.15.53:0/1010011654 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58e39c700 1 --2- 172.21.15.53:0/1010011654 >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 0x7fd58806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.082+0000 7fd58db9b700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd58815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 -- 172.21.15.53:0/1010011654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 msgr2=0x7fd58815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd58815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 -- 172.21.15.53:0/1010011654 >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 msgr2=0x7fd58806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1010011654 >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 0x7fd58806c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 -- 172.21.15.53:0/1010011654 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5881590b0 con 0x7fd58806fb30 2023-11-28T11:19:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58e39c700 1 --2- 172.21.15.53:0/1010011654 >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 0x7fd58806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880cdda0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd58806da30 tx=0x7fd578009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2060a222cee729fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58cb99700 1 -- 172.21.15.53:0/1010011654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800b870 con 0x7fd58806fb30 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58cb99700 1 -- 172.21.15.53:0/1010011654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800b9d0 con 0x7fd58806fb30 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.083+0000 7fd58cb99700 1 -- 172.21.15.53:0/1010011654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800bd60 con 0x7fd58806fb30 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 -- 172.21.15.53:0/1010011654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 msgr2=0x7fd5880cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880cdda0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd58806da30 tx=0x7fd578009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 -- 172.21.15.53:0/1010011654 shutdown_connections 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd58815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1010011654 >> v2:172.21.15.157:3300/0 conn(0x7fd5880ce2e0 0x7fd58806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1010011654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880cdda0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 -- 172.21.15.53:0/1010011654 >> 172.21.15.53:0/1010011654 conn(0x7fd5880cb670 msgr2=0x7fd5880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 -- 172.21.15.53:0/1010011654 shutdown_connections 2023-11-28T11:19:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.084+0000 7fd58fdff700 1 -- 172.21.15.53:0/1010011654 wait complete. 2023-11-28T11:19:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.086+0000 7fd58fdff700 1 Processor -- start 2023-11-28T11:19:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.086+0000 7fd58fdff700 1 -- start start 2023-11-28T11:19:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880ce2e0 0x7fd588145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd588163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588149b50 con 0x7fd58806fb30 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588149540 con 0x7fd5880ce2e0 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd588147350 con 0x7fd588158ce0 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55730/0 (socket says 172.21.15.53:55730) 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 -- 172.21.15.53:0/1471789087 learned_addr learned my addr 172.21.15.53:0/1471789087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58e39c700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd588163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880ce2e0 0x7fd588145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 -- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 msgr2=0x7fd588163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd588163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 -- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880ce2e0 msgr2=0x7fd588145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880ce2e0 0x7fd588145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.087+0000 7fd58db9b700 1 -- 172.21.15.53:0/1471789087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009550 con 0x7fd58806fb30 2023-11-28T11:19:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd58db9b700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd584007e40 tx=0x7fd58400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58400eea0 con 0x7fd58806fb30 2023-11-28T11:19:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584014420 con 0x7fd58806fb30 2023-11-28T11:19:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5840147d0 con 0x7fd58806fb30 2023-11-28T11:19:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880cb3b0 con 0x7fd58806fb30 2023-11-28T11:19:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.088+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5880cb0d0 con 0x7fd58806fb30 2023-11-28T11:19:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.089+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58806b8c0 con 0x7fd58806fb30 2023-11-28T11:19:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.090+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd584012050 con 0x7fd58806fb30 2023-11-28T11:19:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.091+0000 7fd57e7fc700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 0x7fd56c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.091+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd584086000 con 0x7fd58806fb30 2023-11-28T11:19:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.091+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 0x7fd56c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.092+0000 7fd58d39a700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 0x7fd56c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd578000c00 tx=0x7fd578008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.095+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd584052b60 con 0x7fd58806fb30 2023-11-28T11:19:35.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.297+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd58814d7b0 con 0x7fd58806fb30 2023-11-28T11:19:35.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.301+0000 7fd57e7fc700 1 -- 172.21.15.53:0/1471789087 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd584056180 con 0x7fd58806fb30 2023-11-28T11:19:35.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 msgr2=0x7fd56c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 0x7fd56c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd578000c00 tx=0x7fd578008030 comp rx=0 tx=0).stop 2023-11-28T11:19:35.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 msgr2=0x7fd5880ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd584007e40 tx=0x7fd58400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 shutdown_connections 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd56c0682f0 0x7fd56c06a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588158ce0 0x7fd588163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58806fb30 0x7fd5880ae000 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 --2- 172.21.15.53:0/1471789087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880ce2e0 0x7fd588145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 >> 172.21.15.53:0/1471789087 conn(0x7fd5880cb670 msgr2=0x7fd5880ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 shutdown_connections 2023-11-28T11:19:35.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.303+0000 7fd58fdff700 1 -- 172.21.15.53:0/1471789087 wait complete. 2023-11-28T11:19:35.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:19:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.528+0000 7f908a9ff700 1 Processor -- start 2023-11-28T11:19:35.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.528+0000 7f908a9ff700 1 -- start start 2023-11-28T11:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.529+0000 7f908a9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f908415c9b0 0x7f90840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.529+0000 7f908a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90840cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f908a9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90840d1950 con 0x7f9084161340 2023-11-28T11:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f908a9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90840b5800 con 0x7f908415c9b0 2023-11-28T11:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f908a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840ce140 con 0x7f90840cd7c0 2023-11-28T11:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f908415c9b0 0x7f90840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f908415c9b0 0x7f90840b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55746/0 (socket says 172.21.15.53:55746) 2023-11-28T11:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f90837fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9084161340 0x7f90840d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48348/0 (socket says 172.21.15.53:48348) 2023-11-28T11:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 -- 172.21.15.53:0/1087482447 learned_addr learned my addr 172.21.15.53:0/1087482447 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9088f9c700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90840cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 -- 172.21.15.53:0/1087482447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 msgr2=0x7f90840cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90840cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 -- 172.21.15.53:0/1087482447 >> v1:172.21.15.53:6789/0 conn(0x7f9084161340 legacy=0x7f90840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.530+0000 7f9083fff700 1 -- 172.21.15.53:0/1087482447 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840d24f0 con 0x7f908415c9b0 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f9083fff700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908415c9b0 0x7f90840b4bf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f90740098f0 tx=0x7f9074009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2f65bb52c2b736 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f9082ffd700 1 -- 172.21.15.53:0/1087482447 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400b8a0 con 0x7f908415c9b0 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f9082ffd700 1 -- 172.21.15.53:0/1087482447 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907400ba00 con 0x7f908415c9b0 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f9082ffd700 1 -- 172.21.15.53:0/1087482447 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400bd90 con 0x7f908415c9b0 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 -- 172.21.15.53:0/1087482447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908415c9b0 msgr2=0x7f90840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908415c9b0 0x7f90840b4bf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f90740098f0 tx=0x7f9074009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 -- 172.21.15.53:0/1087482447 shutdown_connections 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90840cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 --2- 172.21.15.53:0/1087482447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908415c9b0 0x7f90840b4bf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.531+0000 7f908a9ff700 1 -- 172.21.15.53:0/1087482447 >> 172.21.15.53:0/1087482447 conn(0x7f9084072600 msgr2=0x7f90840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.532+0000 7f908a9ff700 1 -- 172.21.15.53:0/1087482447 shutdown_connections 2023-11-28T11:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.532+0000 7f908a9ff700 1 -- 172.21.15.53:0/1087482447 wait complete. 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.533+0000 7f908a9ff700 1 Processor -- start 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.533+0000 7f908a9ff700 1 -- start start 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90841568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 0x7f908415e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90840c96a0 con 0x7f9084161340 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90840ca2e0 con 0x7f908415c9b0 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.534+0000 7f908a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840c7130 con 0x7f90840cd7c0 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55758/0 (socket says 172.21.15.53:55758) 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 -- 172.21.15.53:0/934626991 learned_addr learned my addr 172.21.15.53:0/934626991 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f90837fe700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 0x7f908415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9083fff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90841568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 -- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 msgr2=0x7f90841568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90841568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 -- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 msgr2=0x7f908415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 0x7f908415e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 -- 172.21.15.53:0/934626991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9074009550 con 0x7f9084161340 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f90837fe700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 0x7f908415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9088f9c700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f907c007e90 tx=0x7f907c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.535+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00ede0 con 0x7f9084161340 2023-11-28T11:19:35.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.536+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c014420 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.536+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0147d0 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.536+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90840d1300 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.536+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90840b5520 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.537+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90840d15e0 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.538+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f907c012050 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.538+0000 7f9080ff9700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 0x7f907006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.538+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f907c086090 con 0x7f9084161340 2023-11-28T11:19:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.539+0000 7f9083fff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 0x7f907006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:35.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.539+0000 7f9083fff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 0x7f907006a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9074014040 tx=0x7f9074008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:35.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.543+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f907c052b20 con 0x7f9084161340 2023-11-28T11:19:35.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.832+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f90840ca4b0 con 0x7f9084161340 2023-11-28T11:19:35.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.852+0000 7f9080ff9700 1 -- 172.21.15.53:0/934626991 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v114) v1 ==== 142+0+0 (secure 0 0 0) 0x7f90840ca4b0 con 0x7f9084161340 2023-11-28T11:19:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 msgr2=0x7f907006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 0x7f907006a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9074014040 tx=0x7f9074008030 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 msgr2=0x7f908415e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f907c007e90 tx=0x7f907c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 shutdown_connections 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f90700682f0 0x7f907006a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840cd7c0 0x7f90841568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9084161340 0x7f908415e940 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 --2- 172.21.15.53:0/934626991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415c9b0 0x7f908415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.854+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 >> 172.21.15.53:0/934626991 conn(0x7f9084072600 msgr2=0x7f908406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.855+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 shutdown_connections 2023-11-28T11:19:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:35.855+0000 7f908a9ff700 1 -- 172.21.15.53:0/934626991 wait complete. 2023-11-28T11:19:35.874 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:19:35.874 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:19:35.874 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:19:35.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:19:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.076+0000 7fa6ebfff700 1 Processor -- start 2023-11-28T11:19:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.076+0000 7fa6ebfff700 1 -- start start 2023-11-28T11:19:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.077+0000 7fa6ebfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6ec0b2560 0x7fa6ec0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.077+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 0x7fa6ec0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.077+0000 7fa6ebfff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0b0500 con 0x7fa6ec0b2010 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.077+0000 7fa6ebfff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b4810 con 0x7fa6ec0b2560 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.077+0000 7fa6ebfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0afbd0 con 0x7fa6ec0b15e0 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6ec0b2560 0x7fa6ec0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6ea7fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa6ec0b2010 0x7fa6ec0b0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48354/0 (socket says 172.21.15.53:48354) 2023-11-28T11:19:36.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6ea7fc700 1 -- 172.21.15.53:0/2612487469 learned_addr learned my addr 172.21.15.53:0/2612487469 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:36.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eb7fe700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 0x7fa6ec0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 -- 172.21.15.53:0/2612487469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 msgr2=0x7fa6ec0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 0x7fa6ec0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 -- 172.21.15.53:0/2612487469 >> v1:172.21.15.53:6789/0 conn(0x7fa6ec0b2010 legacy=0x7fa6ec0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 -- 172.21.15.53:0/2612487469 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0afeb0 con 0x7fa6ec0b2560 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eb7fe700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 0x7fa6ec0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6e9ffb700 1 -- 172.21.15.53:0/2612487469 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999626847 0 0) 0x7fa6ec0b0500 con 0x7fa6ec0b2010 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.078+0000 7fa6eaffd700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0b0990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009840 tx=0x7fa6dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3d9767ae4b6e77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6e9ffb700 1 -- 172.21.15.53:0/2612487469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc011830 con 0x7fa6ec0b2560 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6e9ffb700 1 -- 172.21.15.53:0/2612487469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc011990 con 0x7fa6ec0b2560 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6e9ffb700 1 -- 172.21.15.53:0/2612487469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc011d20 con 0x7fa6ec0b2560 2023-11-28T11:19:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 -- 172.21.15.53:0/2612487469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 msgr2=0x7fa6ec0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0b0990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009840 tx=0x7fa6dc009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 -- 172.21.15.53:0/2612487469 shutdown_connections 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b15e0 0x7fa6ec0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/2612487469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0b0990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 -- 172.21.15.53:0/2612487469 >> 172.21.15.53:0/2612487469 conn(0x7fa6ec09c1d0 msgr2=0x7fa6ec0d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 -- 172.21.15.53:0/2612487469 shutdown_connections 2023-11-28T11:19:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.079+0000 7fa6ebfff700 1 -- 172.21.15.53:0/2612487469 wait complete. 2023-11-28T11:19:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.081+0000 7fa6ebfff700 1 Processor -- start 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.081+0000 7fa6ebfff700 1 -- start start 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0b15e0 0x7fa6ec159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b2010 0x7fa6ec0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15ced0 con 0x7fa6ec0b2560 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15d170 con 0x7fa6ec0b15e0 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6ebfff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15a480 con 0x7fa6ec0b2010 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55780/0 (socket says 172.21.15.53:55780) 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.082+0000 7fa6eb7fe700 1 -- 172.21.15.53:0/1501927100 learned_addr learned my addr 172.21.15.53:0/1501927100 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6ea7fc700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b2010 0x7fa6ec0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eaffd700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0b15e0 0x7fa6ec159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 -- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b2010 msgr2=0x7fa6ec0b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b2010 0x7fa6ec0b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 -- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0b15e0 msgr2=0x7fa6ec159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0b15e0 0x7fa6ec159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 -- 172.21.15.53:0/1501927100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0094a0 con 0x7fa6ec0b2560 2023-11-28T11:19:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6eb7fe700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400cf20 tx=0x7fa6e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400ae80 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4015440 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec0d0070 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40157d0 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.083+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0b0190 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.084+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec0d0200 con 0x7fa6ec0b2560 2023-11-28T11:19:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.085+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6e401c030 con 0x7fa6ec0b2560 2023-11-28T11:19:36.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.086+0000 7fa6d37fe700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 0x7fa6d806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:36.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.086+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa6e4085f00 con 0x7fa6ec0b2560 2023-11-28T11:19:36.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.087+0000 7fa6eaffd700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 0x7fa6d806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:36.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.088+0000 7fa6eaffd700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 0x7fa6d806a7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0090b0 tx=0x7fa6dc01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:36.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.090+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6e4052990 con 0x7fa6ec0b2560 2023-11-28T11:19:36.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.374+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fa6ec061e80 con 0x7fa6ec0b2560 2023-11-28T11:19:36.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.855+0000 7fa6d37fe700 1 -- 172.21.15.53:0/1501927100 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v115) v1 ==== 152+0+0 (secure 0 0 0) 0x7fa6e4055fb0 con 0x7fa6ec0b2560 2023-11-28T11:19:36.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 msgr2=0x7fa6d806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 0x7fa6d806a7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0090b0 tx=0x7fa6dc01c040 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 msgr2=0x7fa6ec0c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400cf20 tx=0x7fa6e400a340 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 shutdown_connections 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa6d8068320 0x7fa6d806a7d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.857+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ec0b2560 0x7fa6ec0c8a00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.858+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6ec0b15e0 0x7fa6ec159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.858+0000 7fa6ebfff700 1 --2- 172.21.15.53:0/1501927100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6ec0b2010 0x7fa6ec0b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.858+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 >> 172.21.15.53:0/1501927100 conn(0x7fa6ec09c1d0 msgr2=0x7fa6ec06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.858+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 shutdown_connections 2023-11-28T11:19:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:36.858+0000 7fa6ebfff700 1 -- 172.21.15.53:0/1501927100 wait complete. 2023-11-28T11:19:36.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:19:37.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.086+0000 7ff1df1ff700 1 Processor -- start 2023-11-28T11:19:37.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.087+0000 7ff1df1ff700 1 -- start start 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1df1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1d80b20b0 0x7ff1d80b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1df1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80b0230 con 0x7ff1d80d1330 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1df1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80b4570 con 0x7ff1d80d01b0 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1df1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815cec0 con 0x7ff1d80b20b0 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1d7fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff1d80d1330 0x7ff1d80b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48358/0 (socket says 172.21.15.53:48358) 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1d7fff700 1 -- 172.21.15.53:0/3752481763 learned_addr learned my addr 172.21.15.53:0/3752481763 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:37.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1dd79c700 1 --2- 172.21.15.53:0/3752481763 >> v2:172.21.15.53:3300/0 conn(0x7ff1d80b20b0 0x7ff1d80b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1dd79c700 1 -- 172.21.15.53:0/3752481763 >> v1:172.21.15.157:6789/0 conn(0x7ff1d80d01b0 legacy=0x7ff1d80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1dd79c700 1 -- 172.21.15.53:0/3752481763 >> v1:172.21.15.53:6789/0 conn(0x7ff1d80d1330 legacy=0x7ff1d80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1dd79c700 1 -- 172.21.15.53:0/3752481763 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80b4850 con 0x7ff1d80b20b0 2023-11-28T11:19:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.088+0000 7ff1d77fe700 1 -- 172.21.15.53:0/3752481763 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660607278 0 0) 0x7ff1d80b0230 con 0x7ff1d80d1330 2023-11-28T11:19:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1dd79c700 1 --2- 172.21.15.53:0/3752481763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80b20b0 0x7ff1d80b0dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009d90 tx=0x7ff1d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0b4bf1ad222ffe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1d77fe700 1 -- 172.21.15.53:0/3752481763 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e070 con 0x7ff1d80b20b0 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1d77fe700 1 -- 172.21.15.53:0/3752481763 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000bbd0 con 0x7ff1d80b20b0 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1d77fe700 1 -- 172.21.15.53:0/3752481763 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0012430 con 0x7ff1d80b20b0 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1df1ff700 1 -- 172.21.15.53:0/3752481763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80b20b0 msgr2=0x7ff1d80b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.089+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/3752481763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80b20b0 0x7ff1d80b0dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009d90 tx=0x7ff1d000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.090+0000 7ff1df1ff700 1 -- 172.21.15.53:0/3752481763 shutdown_connections 2023-11-28T11:19:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.090+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/3752481763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80b20b0 0x7ff1d80b0dd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.090+0000 7ff1df1ff700 1 -- 172.21.15.53:0/3752481763 >> 172.21.15.53:0/3752481763 conn(0x7ff1d809c3f0 msgr2=0x7ff1d80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.090+0000 7ff1df1ff700 1 -- 172.21.15.53:0/3752481763 shutdown_connections 2023-11-28T11:19:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.090+0000 7ff1df1ff700 1 -- 172.21.15.53:0/3752481763 wait complete. 2023-11-28T11:19:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.092+0000 7ff1df1ff700 1 Processor -- start 2023-11-28T11:19:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.092+0000 7ff1df1ff700 1 -- start start 2023-11-28T11:19:37.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.093+0000 7ff1df1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1d80b20b0 0x7ff1d815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1df1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1df1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 0x7ff1d81510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1df1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815fa50 con 0x7ff1d80d01b0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1df1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815fe70 con 0x7ff1d80d1330 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1df1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815edd0 con 0x7ff1d80b20b0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55794/0 (socket says 172.21.15.53:55794) 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 -- 172.21.15.53:0/2257844276 learned_addr learned my addr 172.21.15.53:0/2257844276 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1dd79c700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 0x7ff1d81510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1dcf9b700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1d80b20b0 0x7ff1d815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 -- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1d80b20b0 msgr2=0x7ff1d815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1d80b20b0 0x7ff1d815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 -- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 msgr2=0x7ff1d81510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 0x7ff1d81510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1d7fff700 1 -- 172.21.15.53:0/2257844276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009a70 con 0x7ff1d80d01b0 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.094+0000 7ff1dd79c700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 0x7ff1d81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1d7fff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c450 tx=0x7ff1c400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004c80 con 0x7ff1d80d01b0 2023-11-28T11:19:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c4004e00 con 0x7ff1d80d01b0 2023-11-28T11:19:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d80afec0 con 0x7ff1d80d01b0 2023-11-28T11:19:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4007310 con 0x7ff1d80d01b0 2023-11-28T11:19:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.095+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d80afbe0 con 0x7ff1d80d01b0 2023-11-28T11:19:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.096+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d80b2600 con 0x7ff1d80d01b0 2023-11-28T11:19:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.097+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff1c401a020 con 0x7ff1d80d01b0 2023-11-28T11:19:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.098+0000 7ff1d57fa700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.098+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff1c40855d0 con 0x7ff1d80d01b0 2023-11-28T11:19:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.099+0000 7ff1dcf9b700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.099+0000 7ff1dcf9b700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80ca840 tx=0x7ff1cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.102+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff1c4052000 con 0x7ff1d80d01b0 2023-11-28T11:19:37.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.304+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff1d815fb90 con 0x7ff1d80d01b0 2023-11-28T11:19:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.308+0000 7ff1d57fa700 1 -- 172.21.15.53:0/2257844276 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff1d815fb90 con 0x7ff1d80d01b0 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 msgr2=0x7ff1bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80ca840 tx=0x7ff1cc009040 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 msgr2=0x7ff1d80b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c450 tx=0x7ff1c400c810 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 shutdown_connections 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.310+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1d80b20b0 0x7ff1d815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1d80d1330 0x7ff1d81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 --2- 172.21.15.53:0/2257844276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d80d01b0 0x7ff1d80b82c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 >> 172.21.15.53:0/2257844276 conn(0x7ff1d809c3f0 msgr2=0x7ff1d80b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:37.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 shutdown_connections 2023-11-28T11:19:37.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.311+0000 7ff1df1ff700 1 -- 172.21.15.53:0/2257844276 wait complete. 2023-11-28T11:19:37.331 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:19:37.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:19:37.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:19:37.514 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:19:37.514 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:37.514 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:37.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:19:37.559 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:19:37.559 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:37.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:19:37.605 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:37.605 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:37.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:19:37.649 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:19:37.649 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:37.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:19:37.693 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:19:37.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.898+0000 7f8f8bbff700 1 Processor -- start 2023-11-28T11:19:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.898+0000 7f8f8bbff700 1 -- start start 2023-11-28T11:19:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.899+0000 7f8f8bbff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f840cd7c0 0x7f8f840cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.899+0000 7f8f8bbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840d1950 con 0x7f8f84161340 2023-11-28T11:19:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8bbff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840ce140 con 0x7f8f8415c9b0 2023-11-28T11:19:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8bbff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840b5800 con 0x7f8f840cd7c0 2023-11-28T11:19:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8919a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8f84161340 0x7f8f840d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48372/0 (socket says 172.21.15.53:48372) 2023-11-28T11:19:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8919a700 1 -- 172.21.15.53:0/1206690478 learned_addr learned my addr 172.21.15.53:0/1206690478 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8a19c700 1 --2- 172.21.15.53:0/1206690478 >> v2:172.21.15.157:3300/0 conn(0x7f8f840cd7c0 0x7f8f840cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f88999700 1 -- 172.21.15.53:0/1206690478 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847274592 0 0) 0x7f8f840d1950 con 0x7f8f84161340 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f88999700 1 -- 172.21.15.53:0/1206690478 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f700035b0 con 0x7f8f84161340 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1206690478 >> v1:172.21.15.157:6789/0 conn(0x7f8f8415c9b0 legacy=0x7f8f840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1206690478 >> v1:172.21.15.53:6789/0 conn(0x7f8f84161340 legacy=0x7f8f840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.900+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1206690478 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840d24f0 con 0x7f8f840cd7c0 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f8a19c700 1 --2- 172.21.15.53:0/1206690478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f840cdc00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009d90 tx=0x7f8f7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d4262786dcadd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f88999700 1 -- 172.21.15.53:0/1206690478 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00e070 con 0x7f8f840cd7c0 2023-11-28T11:19:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f88999700 1 -- 172.21.15.53:0/1206690478 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c00bb20 con 0x7f8f840cd7c0 2023-11-28T11:19:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f88999700 1 -- 172.21.15.53:0/1206690478 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c012430 con 0x7f8f840cd7c0 2023-11-28T11:19:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1206690478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 msgr2=0x7f8f840cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.901+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1206690478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f840cdc00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009d90 tx=0x7f8f7c00b330 comp rx=0 tx=0).stop 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.902+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1206690478 shutdown_connections 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.902+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1206690478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f840cdc00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.902+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1206690478 >> 172.21.15.53:0/1206690478 conn(0x7f8f84072600 msgr2=0x7f8f840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.902+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1206690478 shutdown_connections 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.902+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1206690478 wait complete. 2023-11-28T11:19:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.903+0000 7f8f8bbff700 1 Processor -- start 2023-11-28T11:19:37.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.903+0000 7f8f8bbff700 1 -- start start 2023-11-28T11:19:37.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f8414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 0x7f8f841e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84157ba0 con 0x7f8f84161340 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841e7b60 con 0x7f8f840cd7c0 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.904+0000 7f8f8bbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841e7090 con 0x7f8f8415c9b0 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55806/0 (socket says 172.21.15.53:55806) 2023-11-28T11:19:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1901031385 learned_addr learned my addr 172.21.15.53:0/1901031385 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8999b700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f8414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8919a700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 0x7f8f841e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 msgr2=0x7f8f841e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 0x7f8f841e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 msgr2=0x7f8f8414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f8414a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 -- 172.21.15.53:0/1901031385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c009a70 con 0x7f8f84161340 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8999b700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f8414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8919a700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 0x7f8f841e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.905+0000 7f8f8a19c700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f840675d0 tx=0x7f8f7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.906+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c012430 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.906+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c020070 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.906+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f840d1300 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.906+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c004280 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.906+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f840b5520 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.907+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840d15e0 con 0x7f8f84161340 2023-11-28T11:19:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.908+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8f7c0043e0 con 0x7f8f84161340 2023-11-28T11:19:37.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.909+0000 7f8f7a7fc700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 0x7f8f6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:37.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.909+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8f7c022070 con 0x7f8f84161340 2023-11-28T11:19:37.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.909+0000 7f8f8999b700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 0x7f8f6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:37.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.910+0000 7f8f8999b700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 0x7f8f6806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f60 tx=0x7f8f80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:37.913+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8f7c0548a0 con 0x7f8f84161340 2023-11-28T11:19:38.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.162+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f840745c0 con 0x7f8f84161340 2023-11-28T11:19:38.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.162+0000 7f8f7a7fc700 1 -- 172.21.15.53:0/1901031385 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8f7c057ec0 con 0x7f8f84161340 2023-11-28T11:19:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:38.164 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/298981033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":298981033},{"type":"v1","addr":"172.21.15.157:6836","nonce":298981033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4211872271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":4211872271},{"type":"v1","addr":"172.21.15.157:6837","nonce":4211872271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3991108197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3991108197},{"type":"v1","addr":"172.21.15.53:6835","nonce":3991108197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2112242109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2112242109},{"type":"v1","addr":"172.21.15.53:6834","nonce":2112242109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T11:19:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.164+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 msgr2=0x7f8f6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.164+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 0x7f8f6806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f60 tx=0x7f8f80009040 comp rx=0 tx=0).stop 2023-11-28T11:19:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.164+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 msgr2=0x7f8f8415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.164+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f840675d0 tx=0x7f8f7c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 shutdown_connections 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8f680682f0 0x7f8f6806a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f84161340 0x7f8f8415e500 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f840cd7c0 0x7f8f8414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 --2- 172.21.15.53:0/1901031385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f8415c9b0 0x7f8f841e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 >> 172.21.15.53:0/1901031385 conn(0x7f8f84072600 msgr2=0x7f8f8406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 shutdown_connections 2023-11-28T11:19:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.165+0000 7f8f8bbff700 1 -- 172.21.15.53:0/1901031385 wait complete. 2023-11-28T11:19:38.168 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-11-28T11:19:38.185 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:19:38.186 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:19:38.186 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:19:38.186 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:19:38.186 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:19:38.187 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:19:38.187 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:19:38.187 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:19:38.187 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:19:38.188 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:19:38.188 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:19:38.188 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:19:38.188 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:19:38.188 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:19:38.189 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:19:38.189 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:19:38.200 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:19:38.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:19:38.202 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:19:38.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:19:38.204 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:19:38.204 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:19:38.205 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:19:38.205 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:19:38.247 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:19:38.248 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:38.248 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:19:38.248 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:38.250 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:19:38.250 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:38.251 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:19:38.251 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:38.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:19:38.401 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-11-28T11:19:38.401 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-11-28T11:19:38.402 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:19:38.402 INFO:tasks.ceph.mds.d.smithi157.stderr:2023-11-28T11:19:38.401+0000 7ff0273fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T11:19:38.403 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:19:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.472+0000 7ff74f5ff700 1 Processor -- start 2023-11-28T11:19:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.472+0000 7ff74f5ff700 1 -- start start 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.473+0000 7ff74f5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff74815c9b0 0x7ff7480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.473+0000 7ff74f5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff748161340 0x7ff7480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.473+0000 7ff74f5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7480d2320 0x7ff7480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.473+0000 7ff74f5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480b5800 con 0x7ff74815c9b0 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74f5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480d1950 con 0x7ff7480d2320 2023-11-28T11:19:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74f5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b5520 con 0x7ff748161340 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff74815c9b0 0x7ff7480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff74815c9b0 0x7ff7480b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55842/0 (socket says 172.21.15.53:55842) 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 -- 172.21.15.53:0/3940183099 learned_addr learned my addr 172.21.15.53:0/3940183099 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74db9c700 1 --2- 172.21.15.53:0/3940183099 >> v2:172.21.15.157:3300/0 conn(0x7ff7480d2320 0x7ff7480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74cb9a700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff748161340 0x7ff7480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 -- 172.21.15.53:0/3940183099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff748161340 msgr2=0x7ff7480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff748161340 0x7ff7480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 -- 172.21.15.53:0/3940183099 >> v2:172.21.15.157:3300/0 conn(0x7ff7480d2320 msgr2=0x7ff7480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 --2- 172.21.15.53:0/3940183099 >> v2:172.21.15.157:3300/0 conn(0x7ff7480d2320 0x7ff7480cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 -- 172.21.15.53:0/3940183099 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480d1300 con 0x7ff74815c9b0 2023-11-28T11:19:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.474+0000 7ff74d39b700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480b4bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff74808c4d0 tx=0x7ff744009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb9bf487095bf53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:38.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff73ffff700 1 -- 172.21.15.53:0/3940183099 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400b870 con 0x7ff74815c9b0 2023-11-28T11:19:38.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff73ffff700 1 -- 172.21.15.53:0/3940183099 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74400b9d0 con 0x7ff74815c9b0 2023-11-28T11:19:38.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff73ffff700 1 -- 172.21.15.53:0/3940183099 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400bd60 con 0x7ff74815c9b0 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 -- 172.21.15.53:0/3940183099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 msgr2=0x7ff7480b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480b4bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff74808c4d0 tx=0x7ff744009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 -- 172.21.15.53:0/3940183099 shutdown_connections 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/3940183099 >> v2:172.21.15.157:3300/0 conn(0x7ff7480d2320 0x7ff7480cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480b4bf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/3940183099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff748161340 0x7ff7480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 -- 172.21.15.53:0/3940183099 >> 172.21.15.53:0/3940183099 conn(0x7ff748072600 msgr2=0x7ff7480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.475+0000 7ff74f5ff700 1 -- 172.21.15.53:0/3940183099 shutdown_connections 2023-11-28T11:19:38.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.476+0000 7ff74f5ff700 1 -- 172.21.15.53:0/3940183099 wait complete. 2023-11-28T11:19:38.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.477+0000 7ff74f5ff700 1 Processor -- start 2023-11-28T11:19:38.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.477+0000 7ff74f5ff700 1 -- start start 2023-11-28T11:19:38.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 0x7ff7480cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480c9100 con 0x7ff74815c9b0 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480c93e0 con 0x7ff748161340 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74f5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480cafe0 con 0x7ff7480d2320 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.478+0000 7ff74cb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74d39b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74d39b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:49532/0 (socket says 172.21.15.53:49532) 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55854/0 (socket says 172.21.15.53:55854) 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74d39b700 1 -- 172.21.15.53:0/1633168916 learned_addr learned my addr 172.21.15.53:0/1633168916 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74db9c700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 0x7ff7480cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 -- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 msgr2=0x7ff74814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 -- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 msgr2=0x7ff7480cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 0x7ff7480cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 -- 172.21.15.53:0/1633168916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744009550 con 0x7ff74815c9b0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74d39b700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74db9c700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 0x7ff7480cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74cb9a700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e7e0 tx=0x7ff73800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800c070 con 0x7ff74815c9b0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff738013610 con 0x7ff74815c9b0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380139a0 con 0x7ff74815c9b0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.479+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7480b5130 con 0x7ff74815c9b0 2023-11-28T11:19:38.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.480+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7480d15e0 con 0x7ff74815c9b0 2023-11-28T11:19:38.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.481+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7480b52c0 con 0x7ff74815c9b0 2023-11-28T11:19:38.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.482+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff738023070 con 0x7ff74815c9b0 2023-11-28T11:19:38.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.482+0000 7ff73dffb700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:38.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.482+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff7380856a0 con 0x7ff74815c9b0 2023-11-28T11:19:38.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.483+0000 7ff74d39b700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:38.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.483+0000 7ff74d39b700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 0x7ff72c06a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff744009c10 tx=0x7ff744008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:38.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.486+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7380520b0 con 0x7ff74815c9b0 2023-11-28T11:19:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:38.706+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7480cb910 con 0x7ff74815c9b0 2023-11-28T11:19:39.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.429+0000 7ff73dffb700 1 -- 172.21.15.53:0/1633168916 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7380556d0 con 0x7ff74815c9b0 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 msgr2=0x7ff72c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 0x7ff72c06a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff744009c10 tx=0x7ff744008030 comp rx=0 tx=0).stop 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 msgr2=0x7ff7480cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e7e0 tx=0x7ff73800eaf0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 shutdown_connections 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7480d2320 0x7ff74814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748161340 0x7ff7480cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 --2- 172.21.15.53:0/1633168916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74815c9b0 0x7ff7480cd810 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 >> 172.21.15.53:0/1633168916 conn(0x7ff748072600 msgr2=0x7ff7480b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 shutdown_connections 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.432+0000 7ff74f5ff700 1 -- 172.21.15.53:0/1633168916 wait complete. 2023-11-28T11:19:39.434 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-11-28T11:19:39.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:19:39.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.660+0000 7ff3501ff700 1 Processor -- start 2023-11-28T11:19:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.661+0000 7ff3501ff700 1 -- start start 2023-11-28T11:19:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff3501ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff348070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff3501ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34815e1a0 con 0x7ff3480caa70 2023-11-28T11:19:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff3501ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3481585d0 con 0x7ff3480b0d80 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff3501ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34815f070 con 0x7ff348158200 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff34d79a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3480caa70 0x7ff3480b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48398/0 (socket says 172.21.15.53:48398) 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff34d79a700 1 -- 172.21.15.53:0/3215181373 learned_addr learned my addr 172.21.15.53:0/3215181373 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff34df9b700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff348070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.662+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822942407 0 0) 0x7ff34815e1a0 con 0x7ff3480caa70 2023-11-28T11:19:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3340035b0 con 0x7ff3480caa70 2023-11-28T11:19:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1944907004 0 0) 0x7ff3340035b0 con 0x7ff3480caa70 2023-11-28T11:19:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 msgr2=0x7ff348070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff348070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 >> v1:172.21.15.157:6789/0 conn(0x7ff3480b0d80 legacy=0x7ff34815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34815e3f0 con 0x7ff3480caa70 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7ff338001670 con 0x7ff3480caa70 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 >> v1:172.21.15.53:6789/0 conn(0x7ff3480caa70 legacy=0x7ff3480b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff334003a30 0x7ff334003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff334004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff3340050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34815f070 con 0x7ff3340043d0 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33400ba10 con 0x7ff334003a30 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33400bba0 con 0x7ff348158200 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34d79a700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff334004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.663+0000 7ff34df9b700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff334003a30 0x7ff334003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34e79c700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff3340050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 -- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 msgr2=0x7ff3340050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff3340050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 -- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff334003a30 msgr2=0x7ff334003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff334003a30 0x7ff334003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 -- 172.21.15.53:0/3215181373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340035b0 con 0x7ff3340043d0 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34e79c700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff348158200 0x7ff3340050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34d79a700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff334004890 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c240 tx=0x7ff334005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b258fd89179c8c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800cd80 con 0x7ff3340043d0 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff338005960 con 0x7ff3340043d0 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.664+0000 7ff34cf99700 1 -- 172.21.15.53:0/3215181373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338005d10 con 0x7ff3340043d0 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 -- 172.21.15.53:0/3215181373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 msgr2=0x7ff334004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 --2- 172.21.15.53:0/3215181373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff334004890 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c240 tx=0x7ff334005880 comp rx=0 tx=0).stop 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff34e79c700 1 -- 172.21.15.53:0/3215181373 reap_dead start 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 -- 172.21.15.53:0/3215181373 shutdown_connections 2023-11-28T11:19:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 -- 172.21.15.53:0/3215181373 >> 172.21.15.53:0/3215181373 conn(0x7ff3480cb670 msgr2=0x7ff3480cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 -- 172.21.15.53:0/3215181373 shutdown_connections 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.665+0000 7ff3501ff700 1 -- 172.21.15.53:0/3215181373 wait complete. 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.667+0000 7ff3501ff700 1 Processor -- start 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.667+0000 7ff3501ff700 1 -- start start 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3480b37b0 0x7ff3480c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3480c7cf0 0x7ff34814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348071190 con 0x7ff3340043d0 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480b2430 con 0x7ff3480b37b0 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff3501ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480b2710 con 0x7ff3480c7cf0 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff34df9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff34df9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55880/0 (socket says 172.21.15.53:55880) 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff34df9b700 1 -- 172.21.15.53:0/4186467320 learned_addr learned my addr 172.21.15.53:0/4186467320 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff34d79a700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3480b37b0 0x7ff3480c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.668+0000 7ff34e79c700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3480c7cf0 0x7ff34814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 -- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3480c7cf0 msgr2=0x7ff34814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3480c7cf0 0x7ff34814e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 -- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3480b37b0 msgr2=0x7ff3480c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3480b37b0 0x7ff3480c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 -- 172.21.15.53:0/4186467320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34815e9b0 con 0x7ff3340043d0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff34df9b700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff344011aa0 tx=0x7ff344011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344011ed0 con 0x7ff3340043d0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34400f940 con 0x7ff3340043d0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34815e6d0 con 0x7ff3340043d0 2023-11-28T11:19:39.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400fcf0 con 0x7ff3340043d0 2023-11-28T11:19:39.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.669+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34815ec90 con 0x7ff3340043d0 2023-11-28T11:19:39.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.670+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34806fd60 con 0x7ff3340043d0 2023-11-28T11:19:39.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.671+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff344014440 con 0x7ff3340043d0 2023-11-28T11:19:39.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.672+0000 7ff33e7fc700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 0x7ff32c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:39.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.672+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff344086270 con 0x7ff3340043d0 2023-11-28T11:19:39.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.673+0000 7ff34d79a700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 0x7ff32c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:39.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.674+0000 7ff34d79a700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 0x7ff32c06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3380018b0 tx=0x7ff33800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:39.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.676+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff344052d90 con 0x7ff3340043d0 2023-11-28T11:19:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.960+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3480745c0 con 0x7ff3340043d0 2023-11-28T11:19:39.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.961+0000 7ff33e7fc700 1 -- 172.21.15.53:0/4186467320 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7ff3440563b0 con 0x7ff3340043d0 2023-11-28T11:19:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVlNkJINxAAwB6S1pnMrZk3sG6i+8PR9Q==", 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlHGGyFBAAkLNG+TdDzS8Rd+Q2LV6BYw==", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlH5ZYCBAAdBiB6rthc32/VYiX/UUEpg==", 2023-11-28T11:19:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlUdNXIRAAete67odjh+SjYr5KUAmDvw==", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAjzGVlnYF9HxAAx5MdeV6qs7Ce0YL/BzjUuQ==", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAmzGVlheXPJhAAqVOmYx93P2jBYNhzlsg4EQ==", 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQApzGVlRXhLLhAAcCx5wY3VywAx4UWnz15sOg==", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAszGVlxc3CNBAAo8dwkn6w3PZYcNFkTdxZEA==", 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAxzGVlKUZYJRAALIbrf8b2mXrVTt8ebMFSnA==", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA0zGVl2RxXKRAAyHpuvnKEAS2o9BNE1LdRGw==", 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA3zGVlFw1WLRAAf+SrDy5UyYmdT3NfXFYA2A==", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA6zGVlCqCMMBAAq6pur3Ycnp4CiTD40y4upw==", 2023-11-28T11:19:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlvIIFLBAAxrqCs6enHKg93ih+QHQcdw==", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-11-28T11:19:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVlkkNgABAAD7UONIytbl4yWdjvGODhXw==", 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlrXsWNBAA+cEN015xjMYKa6JKavZobQ==", 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVl2rlMCBAA0jfD14UyD2AjgyjtkShrTw==", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAezGVlf022ORAA2Fv9xeE7q2Pkr4B6c7ncyA==", 2023-11-28T11:19:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:19:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-11-28T11:19:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVllCcfLxAAjmT46vM+3CEOyE6aCbidrQ==", 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVl91F6JhAAGc79E5/TfWduB198lkdQqw==", 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:19:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-28T11:19:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 msgr2=0x7ff32c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 0x7ff32c06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3380018b0 tx=0x7ff33800a000 comp rx=0 tx=0).stop 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 msgr2=0x7ff34815c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff344011aa0 tx=0x7ff344011db0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 shutdown_connections 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff32c0682f0 0x7ff32c06a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3480c7cf0 0x7ff34814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3340043d0 0x7ff34815c530 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 --2- 172.21.15.53:0/4186467320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3480b37b0 0x7ff3480c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:39.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 >> 172.21.15.53:0/4186467320 conn(0x7ff3480cb670 msgr2=0x7ff3480712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:39.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 shutdown_connections 2023-11-28T11:19:39.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:39.963+0000 7ff3501ff700 1 -- 172.21.15.53:0/4186467320 wait complete. 2023-11-28T11:19:39.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:40.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.182+0000 7fa7e17ff700 1 Processor -- start 2023-11-28T11:19:40.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.182+0000 7fa7e17ff700 1 -- start start 2023-11-28T11:19:40.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7e17ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7dc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7e17ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0d1950 con 0x7fa7dc161340 2023-11-28T11:19:40.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7e17ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0ce140 con 0x7fa7dc15c9b0 2023-11-28T11:19:40.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7e17ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b5800 con 0x7fa7dc0cd7c0 2023-11-28T11:19:40.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7d3fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa7dc161340 0x7fa7dc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48408/0 (socket says 172.21.15.53:48408) 2023-11-28T11:19:40.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.183+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2322005506 learned_addr learned my addr 172.21.15.53:0/2322005506 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130698626 0 0) 0x7fa7dc0d1950 con 0x7fa7dc161340 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7db7fe700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7dc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c80035b0 con 0x7fa7dc161340 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1039639114 0 0) 0x7fa7c80035b0 con 0x7fa7dc161340 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 msgr2=0x7fa7dc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7dc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 >> v1:172.21.15.157:6789/0 conn(0x7fa7dc15c9b0 legacy=0x7fa7dc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0d24f0 con 0x7fa7dc161340 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa7c4001670 con 0x7fa7dc161340 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 >> v1:172.21.15.53:6789/0 conn(0x7fa7dc161340 legacy=0x7fa7dc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7c8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c8003fb0 0x7fa7c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8004d90 0x7fa7c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b5800 con 0x7fa7c8003fb0 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c800bba0 con 0x7fa7c8004d90 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.184+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c800bd60 con 0x7fa7dc0cd7c0 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c8003fb0 0x7fa7c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7daffd700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7c8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7db7fe700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8004d90 0x7fa7c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 msgr2=0x7fa7c8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc0cd7c0 0x7fa7c8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8004d90 msgr2=0x7fa7c8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8004d90 0x7fa7c8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2322005506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80035b0 con 0x7fa7c8003fb0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c8003fb0 0x7fa7c8004850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c210 tx=0x7fa7c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c5d0d32d0d00ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4007c10 con 0x7fa7c8003fb0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c400cdd0 con 0x7fa7c8003fb0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.185+0000 7fa7da7fc700 1 -- 172.21.15.53:0/2322005506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4005ba0 con 0x7fa7c8003fb0 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2322005506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c8003fb0 msgr2=0x7fa7c8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2322005506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c8003fb0 0x7fa7c8004850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c210 tx=0x7fa7c8005880 comp rx=0 tx=0).stop 2023-11-28T11:19:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7db7fe700 1 -- 172.21.15.53:0/2322005506 reap_dead start 2023-11-28T11:19:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2322005506 shutdown_connections 2023-11-28T11:19:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2322005506 >> 172.21.15.53:0/2322005506 conn(0x7fa7dc072600 msgr2=0x7fa7dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2322005506 shutdown_connections 2023-11-28T11:19:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.186+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2322005506 wait complete. 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.188+0000 7fa7e17ff700 1 Processor -- start 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.188+0000 7fa7e17ff700 1 -- start start 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8003fb0 0x7fa7dc1e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 0x7fa7dc146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc147500 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc1e6c30 con 0x7fa7c8003fb0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7e17ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15ec30 con 0x7fa7dc15e7b0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7d3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7d3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55904/0 (socket says 172.21.15.53:55904) 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2516903282 learned_addr learned my addr 172.21.15.53:0/2516903282 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7daffd700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8003fb0 0x7fa7dc1e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.189+0000 7fa7db7fe700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 0x7fa7dc146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 msgr2=0x7fa7dc146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 0x7fa7dc146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8003fb0 msgr2=0x7fa7dc1e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8003fb0 0x7fa7dc1e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 -- 172.21.15.53:0/2516903282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0d1300 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7db7fe700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 0x7fa7dc146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d3fff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005810 tx=0x7fa7c4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4001630 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c40070d0 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0b5520 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4007480 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.190+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0d15e0 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.191+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc147500 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.192+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7c4010070 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.193+0000 7fa7d37fe700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 0x7fa7bc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.193+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7c4086950 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.193+0000 7fa7daffd700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 0x7fa7bc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.194+0000 7fa7daffd700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 0x7fa7bc06a7d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc15f950 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.197+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7c4053410 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.400+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7dc0745c0 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.401+0000 7fa7d37fe700 1 -- 172.21.15.53:0/2516903282 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14267 (secure 0 0 0) 0x7fa7c402b090 con 0x7fa7dc0c85d0 2023-11-28T11:19:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:40.402 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:39.424750+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.403+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 msgr2=0x7fa7bc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.403+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 0x7fa7bc06a7d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc15f950 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 msgr2=0x7fa7dc15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005810 tx=0x7fa7c4001a00 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 shutdown_connections 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa7bc068320 0x7fa7bc06a7d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7dc15e7b0 0x7fa7dc146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c8003fb0 0x7fa7dc1e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 --2- 172.21.15.53:0/2516903282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7dc0c85d0 0x7fa7dc15e270 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 >> 172.21.15.53:0/2516903282 conn(0x7fa7dc072600 msgr2=0x7fa7dc0b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 shutdown_connections 2023-11-28T11:19:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.404+0000 7fa7e17ff700 1 -- 172.21.15.53:0/2516903282 wait complete. 2023-11-28T11:19:40.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:19:40.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:19:40.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.627+0000 7f6813bff700 1 Processor -- start 2023-11-28T11:19:40.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.627+0000 7f6813bff700 1 -- start start 2023-11-28T11:19:40.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f6813bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f680c161340 0x7f680c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f6813bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f6813bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0d1950 con 0x7f680c15c9b0 2023-11-28T11:19:40.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f6813bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c0b5800 con 0x7f680c161340 2023-11-28T11:19:40.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f6813bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0b5300 con 0x7f680c0d2320 2023-11-28T11:19:40.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f681199b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f680c15c9b0 0x7f680c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:37714/0 (socket says 172.21.15.53:37714) 2023-11-28T11:19:40.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f681219c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f681199b700 1 -- 172.21.15.53:0/998238381 learned_addr learned my addr 172.21.15.53:0/998238381 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:40.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.628+0000 7f681119a700 1 --2- 172.21.15.53:0/998238381 >> v2:172.21.15.157:3300/0 conn(0x7f680c161340 0x7f680c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f681119a700 1 -- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 msgr2=0x7f680c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f681119a700 1 --2- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f6810999700 1 -- 172.21.15.53:0/998238381 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290926103 0 0) 0x7f680c0d1950 con 0x7f680c15c9b0 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f681119a700 1 -- 172.21.15.53:0/998238381 >> v1:172.21.15.157:6789/0 conn(0x7f680c15c9b0 legacy=0x7f680c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f681119a700 1 -- 172.21.15.53:0/998238381 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0b5520 con 0x7f680c161340 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.629+0000 7f681119a700 1 --2- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0d1de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009840 tx=0x7f67fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be32d07afb62883 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6810999700 1 -- 172.21.15.53:0/998238381 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc011830 con 0x7f680c161340 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6810999700 1 -- 172.21.15.53:0/998238381 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc011990 con 0x7f680c161340 2023-11-28T11:19:40.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6810999700 1 -- 172.21.15.53:0/998238381 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc011d20 con 0x7f680c161340 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 -- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 msgr2=0x7f680c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 --2- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0d1de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009840 tx=0x7f67fc009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 -- 172.21.15.53:0/998238381 shutdown_connections 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 --2- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 --2- 172.21.15.53:0/998238381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0d1de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 -- 172.21.15.53:0/998238381 >> 172.21.15.53:0/998238381 conn(0x7f680c072600 msgr2=0x7f680c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:40.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 -- 172.21.15.53:0/998238381 shutdown_connections 2023-11-28T11:19:40.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.630+0000 7f6813bff700 1 -- 172.21.15.53:0/998238381 wait complete. 2023-11-28T11:19:40.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.632+0000 7f6813bff700 1 Processor -- start 2023-11-28T11:19:40.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.632+0000 7f6813bff700 1 -- start start 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c148210 con 0x7f680c15c9b0 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c148440 con 0x7f680c161340 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f6813bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c148720 con 0x7f680c0d2320 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f681119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f681119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55916/0 (socket says 172.21.15.53:55916) 2023-11-28T11:19:40.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.633+0000 7f681119a700 1 -- 172.21.15.53:0/2850919861 learned_addr learned my addr 172.21.15.53:0/2850919861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681199b700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681219c700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 -- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 msgr2=0x7f680c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 -- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 msgr2=0x7f680c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 -- 172.21.15.53:0/2850919861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0094a0 con 0x7f680c15c9b0 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681199b700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681219c700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:40.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f681119a700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009fd0 tx=0x7f67fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0038a0 con 0x7f680c15c9b0 2023-11-28T11:19:40.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.634+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc01f070 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.635+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc003c40 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.635+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c0d15e0 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.635+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c0d1300 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.636+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c15cd80 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.637+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67fc00f050 con 0x7f680c15c9b0 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.637+0000 7f68027fc700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 0x7f67f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:40.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.638+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f67fc021070 con 0x7f680c15c9b0 2023-11-28T11:19:40.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.638+0000 7f681199b700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 0x7f67f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:40.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.639+0000 7f681199b700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 0x7f67f006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6808000f60 tx=0x7f6808007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:40.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.642+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f67fc053ea0 con 0x7f680c15c9b0 2023-11-28T11:19:40.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.843+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f680c0745c0 con 0x7f680c15c9b0 2023-11-28T11:19:40.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.843+0000 7f68027fc700 1 -- 172.21.15.53:0/2850919861 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67fc010380 con 0x7f680c15c9b0 2023-11-28T11:19:40.844 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 msgr2=0x7f67f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 0x7f67f006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6808000f60 tx=0x7f6808007d50 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 msgr2=0x7f680c147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009fd0 tx=0x7f67fc004750 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 shutdown_connections 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f67f00682f0 0x7f67f006a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f680c0d2320 0x7f680c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f680c161340 0x7f680c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 --2- 172.21.15.53:0/2850919861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f680c15c9b0 0x7f680c147cd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:40.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 >> 172.21.15.53:0/2850919861 conn(0x7f680c072600 msgr2=0x7f680c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:40.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 shutdown_connections 2023-11-28T11:19:40.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:40.846+0000 7f6813bff700 1 -- 172.21.15.53:0/2850919861 wait complete. 2023-11-28T11:19:40.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:19:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.071+0000 7f2c53dff700 1 Processor -- start 2023-11-28T11:19:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.071+0000 7f2c53dff700 1 -- start start 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c53dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c53dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c4c161340 0x7f2c4c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c53dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0d1950 con 0x7f2c4c0d2320 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c53dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b5800 con 0x7f2c4c161340 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c53dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b5520 con 0x7f2c4c15c9b0 2023-11-28T11:19:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c5239c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2c4c0d2320 0x7f2c4c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56132/0 (socket says 172.21.15.53:56132) 2023-11-28T11:19:41.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c5239c700 1 -- 172.21.15.53:0/2275156368 learned_addr learned my addr 172.21.15.53:0/2275156368 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:41.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c5139a700 1 --2- 172.21.15.53:0/2275156368 >> v2:172.21.15.157:3300/0 conn(0x7f2c4c161340 0x7f2c4c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.072+0000 7f2c51b9b700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c50b99700 1 -- 172.21.15.53:0/2275156368 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462585536 0 0) 0x7f2c4c0d1950 con 0x7f2c4c0d2320 2023-11-28T11:19:41.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c50b99700 1 -- 172.21.15.53:0/2275156368 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c380035b0 con 0x7f2c4c0d2320 2023-11-28T11:19:41.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c5139a700 1 -- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 msgr2=0x7f2c4c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c5139a700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c5139a700 1 -- 172.21.15.53:0/2275156368 >> v1:172.21.15.53:6789/0 conn(0x7f2c4c0d2320 legacy=0x7f2c4c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c5139a700 1 -- 172.21.15.53:0/2275156368 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0d1300 con 0x7f2c4c161340 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c51b9b700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.073+0000 7f2c5139a700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c161340 0x7f2c4c0d1de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009840 tx=0x7f2c3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96730ed05ddb6ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c50b99700 1 -- 172.21.15.53:0/2275156368 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c011830 con 0x7f2c4c161340 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c50b99700 1 -- 172.21.15.53:0/2275156368 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c011990 con 0x7f2c4c161340 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c50b99700 1 -- 172.21.15.53:0/2275156368 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c011d20 con 0x7f2c4c161340 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 -- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c161340 msgr2=0x7f2c4c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c161340 0x7f2c4c0d1de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009840 tx=0x7f2c3c009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 -- 172.21.15.53:0/2275156368 shutdown_connections 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 --2- 172.21.15.53:0/2275156368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c161340 0x7f2c4c0d1de0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 -- 172.21.15.53:0/2275156368 >> 172.21.15.53:0/2275156368 conn(0x7f2c4c072600 msgr2=0x7f2c4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 -- 172.21.15.53:0/2275156368 shutdown_connections 2023-11-28T11:19:41.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.074+0000 7f2c53dff700 1 -- 172.21.15.53:0/2275156368 wait complete. 2023-11-28T11:19:41.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.077+0000 7f2c53dff700 1 Processor -- start 2023-11-28T11:19:41.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.077+0000 7f2c53dff700 1 -- start start 2023-11-28T11:19:41.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c0d2320 0x7f2c4c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c151b90 con 0x7f2c4c161340 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c151610 con 0x7f2c4c0d2320 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c53dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b6140 con 0x7f2c4c15c9b0 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c5239c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c5239c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59352/0 (socket says 172.21.15.53:59352) 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.078+0000 7f2c5239c700 1 -- 172.21.15.53:0/4097929393 learned_addr learned my addr 172.21.15.53:0/4097929393 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5139a700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c51b9b700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c0d2320 0x7f2c4c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 -- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 msgr2=0x7f2c4c0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 -- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c0d2320 msgr2=0x7f2c4c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c0d2320 0x7f2c4c0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 -- 172.21.15.53:0/4097929393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480094a0 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c5239c700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400caa0 tx=0x7f2c4400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400aa80 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4400abe0 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c0094a0 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44012400 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.079+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c0b5130 con 0x7f2c4c161340 2023-11-28T11:19:41.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.081+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c0d15e0 con 0x7f2c4c161340 2023-11-28T11:19:41.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.081+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c44019030 con 0x7f2c4c161340 2023-11-28T11:19:41.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.082+0000 7f2c427fc700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 0x7f2c3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.082+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2c440855d0 con 0x7f2c4c161340 2023-11-28T11:19:41.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.083+0000 7f2c51b9b700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 0x7f2c3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.083+0000 7f2c51b9b700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 0x7f2c3006a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d00 tx=0x7f2c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.087+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c44052120 con 0x7f2c4c161340 2023-11-28T11:19:41.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.288+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c4c0b52c0 con 0x7f2c4c161340 2023-11-28T11:19:41.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.288+0000 7f2c427fc700 1 -- 172.21.15.53:0/4097929393 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c44017020 con 0x7f2c4c161340 2023-11-28T11:19:41.289 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 msgr2=0x7f2c3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 0x7f2c3006a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d00 tx=0x7f2c48008040 comp rx=0 tx=0).stop 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 msgr2=0x7f2c4c0c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400caa0 tx=0x7f2c4400ce60 comp rx=0 tx=0).stop 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 shutdown_connections 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2c300682f0 0x7f2c3006a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c161340 0x7f2c4c0c8c00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c4c0d2320 0x7f2c4c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 --2- 172.21.15.53:0/4097929393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4c15c9b0 0x7f2c4c0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 >> 172.21.15.53:0/4097929393 conn(0x7f2c4c072600 msgr2=0x7f2c4c160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 shutdown_connections 2023-11-28T11:19:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.291+0000 7f2c53dff700 1 -- 172.21.15.53:0/4097929393 wait complete. 2023-11-28T11:19:41.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:19:41.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.511+0000 7f650c7ff700 1 Processor -- start 2023-11-28T11:19:41.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.511+0000 7f650c7ff700 1 -- start start 2023-11-28T11:19:41.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6504158200 0x7f6504070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caa70 0x7f65040b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65040caf10 0x7f65040b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650415f070 con 0x7f65040caf10 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6504071190 con 0x7f6504158200 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650c7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650415e1a0 con 0x7f65040caa70 2023-11-28T11:19:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650ad9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65040caf10 0x7f65040b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650ad9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65040caf10 0x7f65040b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59362/0 (socket says 172.21.15.53:59362) 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650ad9c700 1 -- 172.21.15.53:0/2120822417 learned_addr learned my addr 172.21.15.53:0/2120822417 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f6509d9a700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caa70 0x7f65040b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.512+0000 7f650a59b700 1 --2- 172.21.15.53:0/2120822417 >> v2:172.21.15.157:3300/0 conn(0x7f6504158200 0x7f6504070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 -- 172.21.15.53:0/2120822417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caa70 msgr2=0x7f65040b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caa70 0x7f65040b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 -- 172.21.15.53:0/2120822417 >> v2:172.21.15.157:3300/0 conn(0x7f6504158200 msgr2=0x7f6504070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2120822417 >> v2:172.21.15.157:3300/0 conn(0x7f6504158200 0x7f6504070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 -- 172.21.15.53:0/2120822417 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65041585d0 con 0x7f65040caf10 2023-11-28T11:19:41.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65040caf10 0x7f65040b0bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce9b0 tx=0x7f64fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e873a59e5f549c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f6509599700 1 -- 172.21.15.53:0/2120822417 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00e070 con 0x7f65040caf10 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f6509599700 1 -- 172.21.15.53:0/2120822417 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc00b8d0 con 0x7f65040caf10 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.513+0000 7f6509599700 1 -- 172.21.15.53:0/2120822417 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00bc80 con 0x7f65040caf10 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 -- 172.21.15.53:0/2120822417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65040caf10 msgr2=0x7f65040b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65040caf10 0x7f65040b0bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce9b0 tx=0x7f64fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 -- 172.21.15.53:0/2120822417 shutdown_connections 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65040caf10 0x7f65040b0bb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2120822417 >> v2:172.21.15.157:3300/0 conn(0x7f6504158200 0x7f6504070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2120822417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caa70 0x7f65040b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 -- 172.21.15.53:0/2120822417 >> 172.21.15.53:0/2120822417 conn(0x7f65040cb670 msgr2=0x7f65040cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 -- 172.21.15.53:0/2120822417 shutdown_connections 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.514+0000 7f650c7ff700 1 -- 172.21.15.53:0/2120822417 wait complete. 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.516+0000 7f650c7ff700 1 Processor -- start 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.516+0000 7f650c7ff700 1 -- start start 2023-11-28T11:19:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65040caa70 0x7f6504156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 0x7f65040cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65041574b0 con 0x7f6504158200 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b3060 con 0x7f65040caa70 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650c7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040ad840 con 0x7f65040caf10 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59368/0 (socket says 172.21.15.53:59368) 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 -- 172.21.15.53:0/2013711536 learned_addr learned my addr 172.21.15.53:0/2013711536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650a59b700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65040caa70 0x7f6504156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f6509d9a700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 0x7f65040cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 -- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 msgr2=0x7f65040cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 0x7f65040cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 -- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65040caa70 msgr2=0x7f6504156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65040caa70 0x7f6504156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f650ad9c700 1 -- 172.21.15.53:0/2013711536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009b30 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.517+0000 7f6509d9a700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 0x7f65040cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f650ad9c700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6504067360 tx=0x7f64fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004930 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc0092e0 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01c680 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650415e6d0 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.518+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650415e3f0 con 0x7f6504158200 2023-11-28T11:19:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.519+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650415e9b0 con 0x7f6504158200 2023-11-28T11:19:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.520+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f64fc02c070 con 0x7f6504158200 2023-11-28T11:19:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.521+0000 7f64faffd700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 0x7f64e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.521+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f64fc087490 con 0x7f6504158200 2023-11-28T11:19:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.521+0000 7f650a59b700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 0x7f64e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.522+0000 7f650a59b700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 0x7f64e806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6500009ac0 tx=0x7f6500009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.525+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f64fc054040 con 0x7f6504158200 2023-11-28T11:19:41.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.729+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f650406fd60 con 0x7f6504158200 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.730+0000 7f64faffd700 1 -- 172.21.15.53:0/2013711536 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+126 (secure 0 0 0) 0x7f64fc057660 con 0x7f6504158200 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-11-28T11:19:41.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:41.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:41.732 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 2, 2023-11-28T11:19:41.732 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "rbd" 2023-11-28T11:19:41.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:41.732 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 msgr2=0x7f64e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 0x7f64e806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6500009ac0 tx=0x7f6500009210 comp rx=0 tx=0).stop 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 msgr2=0x7f65040cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6504067360 tx=0x7f64fc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 shutdown_connections 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64e80682f0 0x7f64e806a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6504158200 0x7f65040cd560 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65040caa70 0x7f6504156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 --2- 172.21.15.53:0/2013711536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65040caf10 0x7f65040cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 >> 172.21.15.53:0/2013711536 conn(0x7f65040cb670 msgr2=0x7f650415d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:41.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.732+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 shutdown_connections 2023-11-28T11:19:41.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.733+0000 7f650c7ff700 1 -- 172.21.15.53:0/2013711536 wait complete. 2023-11-28T11:19:41.752 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:19:41.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:19:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.959+0000 7f71471ff700 1 Processor -- start 2023-11-28T11:19:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.959+0000 7f71471ff700 1 -- start start 2023-11-28T11:19:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.960+0000 7f71471ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7140161200 0x7f714015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.960+0000 7f71471ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7140160d70 con 0x7f714015a760 2023-11-28T11:19:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.960+0000 7f71471ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400b5390 con 0x7f71400cded0 2023-11-28T11:19:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.960+0000 7f71471ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714015ab30 con 0x7f7140161200 2023-11-28T11:19:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f7144f9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f714015a760 0x7f71400cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56146/0 (socket says 172.21.15.53:56146) 2023-11-28T11:19:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f7144f9b700 1 -- 172.21.15.53:0/4017434271 learned_addr learned my addr 172.21.15.53:0/4017434271 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f714579c700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7140161200 0x7f714015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714191319 0 0) 0x7f7140160d70 con 0x7f714015a760 2023-11-28T11:19:41.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71300035b0 con 0x7f714015a760 2023-11-28T11:19:41.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236832084 0 0) 0x7f71400b5390 con 0x7f71400cded0 2023-11-28T11:19:41.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7140160d70 con 0x7f71400cded0 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2186431823 0 0) 0x7f71300035b0 con 0x7f714015a760 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7140161200 msgr2=0x7f714015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7140161200 0x7f714015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 >> v1:172.21.15.157:6789/0 conn(0x7f71400cded0 legacy=0x7f71400d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400b5670 con 0x7f714015a760 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f7134001670 con 0x7f714015a760 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 >> v1:172.21.15.53:6789/0 conn(0x7f714015a760 legacy=0x7f71400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.961+0000 7f713f7fe700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130003bc0 0x7f7130003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713f7fe700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7130004850 0x7f7130004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713f7fe700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7130005200 0x7f713000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714015ab30 con 0x7f7130004850 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713000bfd0 con 0x7f7130005200 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713000c190 con 0x7f7130003bc0 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f714579c700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7140161200 0x7f714015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7130004850 0x7f7130004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f7144f9b700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130003bc0 0x7f7130003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f714579c700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7130005200 0x7f713000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 -- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130003bc0 msgr2=0x7f7130003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130003bc0 0x7f7130003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 -- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7130005200 msgr2=0x7f713000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7130005200 0x7f713000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f713ffff700 1 -- 172.21.15.53:0/4017434271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300035b0 con 0x7f7130004850 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.962+0000 7f714579c700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7130005200 0x7f713000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:41.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f713ffff700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7130004850 0x7f7130004cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f71400ce2c0 tx=0x7f7130005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7d2b2649449219 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c001470 con 0x7f7130004850 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c0050d0 con 0x7f7130004850 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f713f7fe700 1 -- 172.21.15.53:0/4017434271 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c005460 con 0x7f7130004850 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 -- 172.21.15.53:0/4017434271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7130004850 msgr2=0x7f7130004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 --2- 172.21.15.53:0/4017434271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7130004850 0x7f7130004cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f71400ce2c0 tx=0x7f7130005880 comp rx=0 tx=0).stop 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f714579c700 1 -- 172.21.15.53:0/4017434271 reap_dead start 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 -- 172.21.15.53:0/4017434271 shutdown_connections 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 -- 172.21.15.53:0/4017434271 >> 172.21.15.53:0/4017434271 conn(0x7f7140072620 msgr2=0x7f71400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 -- 172.21.15.53:0/4017434271 shutdown_connections 2023-11-28T11:19:41.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.963+0000 7f71471ff700 1 -- 172.21.15.53:0/4017434271 wait complete. 2023-11-28T11:19:41.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.965+0000 7f71471ff700 1 Processor -- start 2023-11-28T11:19:41.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.965+0000 7f71471ff700 1 -- start start 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 0x7f71400b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 0x7f71401f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c7fc0 con 0x7f714014a8c0 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c82a0 con 0x7f714014a450 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.966+0000 7f71471ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c8580 con 0x7f7130004850 2023-11-28T11:19:41.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59386/0 (socket says 172.21.15.53:59386) 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 -- 172.21.15.53:0/3059444562 learned_addr learned my addr 172.21.15.53:0/3059444562 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f7144f9b700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 0x7f71400b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f713ffff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 0x7f71401f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 -- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 msgr2=0x7f71400b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 0x7f71400b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 -- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 msgr2=0x7f71401f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 0x7f71401f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 -- 172.21.15.53:0/3059444562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140160a00 con 0x7f714014a8c0 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f7144f9b700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 0x7f71400b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f713ffff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 0x7f71401f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:41.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.967+0000 7f714579c700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71400cf200 tx=0x7f7138011790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.968+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800e070 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.968+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138011ea0 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.968+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7140160720 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.968+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138009ea0 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.968+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714015dc30 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.969+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714004e620 con 0x7f714014a8c0 2023-11-28T11:19:41.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.970+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7138003a20 con 0x7f714014a8c0 2023-11-28T11:19:41.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.971+0000 7f713d7fa700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 0x7f712406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:41.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.971+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7138086250 con 0x7f714014a8c0 2023-11-28T11:19:41.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.971+0000 7f7144f9b700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 0x7f712406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:41.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.972+0000 7f7144f9b700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 0x7f712406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:41.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:41.975+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7138052da0 con 0x7f714014a8c0 2023-11-28T11:19:42.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.177+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f71400d08a0 con 0x7f714014a8c0 2023-11-28T11:19:42.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.882+0000 7f713d7fa700 1 -- 172.21.15.53:0/3059444562 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7138003cb0 con 0x7f714014a8c0 2023-11-28T11:19:42.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.889+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 msgr2=0x7f712406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:42.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.889+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 0x7f712406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134009040 comp rx=0 tx=0).stop 2023-11-28T11:19:42.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.889+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 msgr2=0x7f714006a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:42.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.889+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71400cf200 tx=0x7f7138011790 comp rx=0 tx=0).stop 2023-11-28T11:19:42.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 shutdown_connections 2023-11-28T11:19:42.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f71240682f0 0x7f712406a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:42.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7130004850 0x7f71400b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:42.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f714014a8c0 0x7f714006a900 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:42.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 --2- 172.21.15.53:0/3059444562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714014a450 0x7f71401f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:42.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 >> 172.21.15.53:0/3059444562 conn(0x7f7140072620 msgr2=0x7f714015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:42.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.891+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 shutdown_connections 2023-11-28T11:19:42.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:42.892+0000 7f71471ff700 1 -- 172.21.15.53:0/3059444562 wait complete. 2023-11-28T11:19:42.894 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-11-28T11:19:42.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:19:43.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.129+0000 7fdc449ff700 1 Processor -- start 2023-11-28T11:19:43.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.129+0000 7fdc449ff700 1 -- start start 2023-11-28T11:19:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 0x7fdc400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc40161340 0x7fdc400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 0x7fdc400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400b5800 con 0x7fdc40161340 2023-11-28T11:19:43.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400d1950 con 0x7fdc400d2320 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc449ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b5520 con 0x7fdc4015c9b0 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc37fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc40161340 0x7fdc400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc37fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc40161340 0x7fdc400d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59398/0 (socket says 172.21.15.53:59398) 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc37fff700 1 -- 172.21.15.53:0/2354091629 learned_addr learned my addr 172.21.15.53:0/2354091629 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc3e59c700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 0x7fdc400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.130+0000 7fdc3ed9d700 1 --2- 172.21.15.53:0/2354091629 >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 0x7fdc400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 -- 172.21.15.53:0/2354091629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 msgr2=0x7fdc400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 0x7fdc400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 -- 172.21.15.53:0/2354091629 >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 msgr2=0x7fdc400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 --2- 172.21.15.53:0/2354091629 >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 0x7fdc400cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 -- 172.21.15.53:0/2354091629 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400d1300 con 0x7fdc40161340 2023-11-28T11:19:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc3e59c700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 0x7fdc400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc3ed9d700 1 --2- 172.21.15.53:0/2354091629 >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 0x7fdc400cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc37fff700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc40161340 0x7fdc400d1de0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdc4008c4d0 tx=0x7fdc28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9da83f52d1e6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc3dd9b700 1 -- 172.21.15.53:0/2354091629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800b870 con 0x7fdc40161340 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc3dd9b700 1 -- 172.21.15.53:0/2354091629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800b9d0 con 0x7fdc40161340 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.131+0000 7fdc3dd9b700 1 -- 172.21.15.53:0/2354091629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800bd60 con 0x7fdc40161340 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 -- 172.21.15.53:0/2354091629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc40161340 msgr2=0x7fdc400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc40161340 0x7fdc400d1de0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdc4008c4d0 tx=0x7fdc28009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 -- 172.21.15.53:0/2354091629 shutdown_connections 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc4015c9b0 0x7fdc400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 --2- 172.21.15.53:0/2354091629 >> v2:172.21.15.157:3300/0 conn(0x7fdc400d2320 0x7fdc400cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 --2- 172.21.15.53:0/2354091629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc40161340 0x7fdc400d1de0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 -- 172.21.15.53:0/2354091629 >> 172.21.15.53:0/2354091629 conn(0x7fdc40072600 msgr2=0x7fdc400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 -- 172.21.15.53:0/2354091629 shutdown_connections 2023-11-28T11:19:43.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.132+0000 7fdc449ff700 1 -- 172.21.15.53:0/2354091629 wait complete. 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.134+0000 7fdc449ff700 1 Processor -- start 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.134+0000 7fdc449ff700 1 -- start start 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 0x7fdc400c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 0x7fdc401536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40150940 con 0x7fdc400d2320 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40154bb0 con 0x7fdc4015c9b0 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc449ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40154e50 con 0x7fdc40161340 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc3e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc3e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59402/0 (socket says 172.21.15.53:59402) 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc3e59c700 1 -- 172.21.15.53:0/3757222994 learned_addr learned my addr 172.21.15.53:0/3757222994 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc3e59c700 1 -- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 msgr2=0x7fdc401536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc3ed9d700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 0x7fdc401536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.135+0000 7fdc37fff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 0x7fdc400c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3e59c700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 0x7fdc401536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3e59c700 1 -- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 msgr2=0x7fdc400c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3e59c700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 0x7fdc400c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3e59c700 1 -- 172.21.15.53:0/3757222994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28009550 con 0x7fdc400d2320 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc37fff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 0x7fdc400c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3ed9d700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 0x7fdc401536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc3e59c700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000e760 tx=0x7fdc3000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30013610 con 0x7fdc400d2320 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc30013770 con 0x7fdc400d2320 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30013b20 con 0x7fdc400d2320 2023-11-28T11:19:43.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc400b5130 con 0x7fdc400d2320 2023-11-28T11:19:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.136+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400d15e0 con 0x7fdc400d2320 2023-11-28T11:19:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.137+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc400b52c0 con 0x7fdc400d2320 2023-11-28T11:19:43.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.138+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc30023070 con 0x7fdc400d2320 2023-11-28T11:19:43.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.139+0000 7fdc36ffd700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 0x7fdc2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:43.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.140+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fdc300856e0 con 0x7fdc400d2320 2023-11-28T11:19:43.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.140+0000 7fdc37fff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 0x7fdc2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:43.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.141+0000 7fdc37fff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 0x7fdc2006a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdc401f5f20 tx=0x7fdc28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:43.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.144+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdc30052030 con 0x7fdc400d2320 2023-11-28T11:19:43.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.347+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdc400745c0 con 0x7fdc400d2320 2023-11-28T11:19:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.890+0000 7fdc36ffd700 1 -- 172.21.15.53:0/3757222994 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdc30055650 con 0x7fdc400d2320 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.899+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 msgr2=0x7fdc2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.899+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 0x7fdc2006a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdc401f5f20 tx=0x7fdc28008030 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.899+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 msgr2=0x7fdc40150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.899+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000e760 tx=0x7fdc3000eb20 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 shutdown_connections 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdc200682f0 0x7fdc2006a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc40161340 0x7fdc401536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc400d2320 0x7fdc40150400 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 --2- 172.21.15.53:0/3757222994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4015c9b0 0x7fdc400c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.900+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 >> 172.21.15.53:0/3757222994 conn(0x7fdc40072600 msgr2=0x7fdc40160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:43.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.901+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 shutdown_connections 2023-11-28T11:19:43.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:43.901+0000 7fdc449ff700 1 -- 172.21.15.53:0/3757222994 wait complete. 2023-11-28T11:19:43.907 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-11-28T11:19:43.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:19:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.136+0000 7f95e2bff700 1 Processor -- start 2023-11-28T11:19:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.137+0000 7f95e2bff700 1 -- start start 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e2bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95dc15c9b0 0x7f95dc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e2bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95dc161340 0x7f95dc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e2bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0d1950 con 0x7f95dc0d2320 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e2bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc0b5800 con 0x7f95dc161340 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e2bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc0b5520 con 0x7f95dc15c9b0 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95dbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95dc161340 0x7f95dc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e119c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f95dc0d2320 0x7f95dc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56160/0 (socket says 172.21.15.53:56160) 2023-11-28T11:19:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.138+0000 7f95e119c700 1 -- 172.21.15.53:0/3971465306 learned_addr learned my addr 172.21.15.53:0/3971465306 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95e099b700 1 --2- 172.21.15.53:0/3971465306 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15c9b0 0x7f95dc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95dbfff700 1 -- 172.21.15.53:0/3971465306 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15c9b0 msgr2=0x7f95dc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95dbfff700 1 --2- 172.21.15.53:0/3971465306 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15c9b0 0x7f95dc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95dbfff700 1 -- 172.21.15.53:0/3971465306 >> v1:172.21.15.53:6789/0 conn(0x7f95dc0d2320 legacy=0x7f95dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95dbfff700 1 -- 172.21.15.53:0/3971465306 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0d1300 con 0x7f95dc161340 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95db7fe700 1 -- 172.21.15.53:0/3971465306 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864792631 0 0) 0x7f95dc0d1950 con 0x7f95dc0d2320 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95dbfff700 1 --2- 172.21.15.53:0/3971465306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc161340 0x7f95dc0d1de0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95dc067460 tx=0x7f95c8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b645f60a80dbf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95db7fe700 1 -- 172.21.15.53:0/3971465306 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8011830 con 0x7f95dc161340 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95db7fe700 1 -- 172.21.15.53:0/3971465306 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8011990 con 0x7f95dc161340 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.139+0000 7f95db7fe700 1 -- 172.21.15.53:0/3971465306 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8011d20 con 0x7f95dc161340 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 -- 172.21.15.53:0/3971465306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc161340 msgr2=0x7f95dc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 --2- 172.21.15.53:0/3971465306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc161340 0x7f95dc0d1de0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95dc067460 tx=0x7f95c8009b20 comp rx=0 tx=0).stop 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 -- 172.21.15.53:0/3971465306 shutdown_connections 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 --2- 172.21.15.53:0/3971465306 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15c9b0 0x7f95dc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 --2- 172.21.15.53:0/3971465306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc161340 0x7f95dc0d1de0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 -- 172.21.15.53:0/3971465306 >> 172.21.15.53:0/3971465306 conn(0x7f95dc072600 msgr2=0x7f95dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 -- 172.21.15.53:0/3971465306 shutdown_connections 2023-11-28T11:19:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.140+0000 7f95e2bff700 1 -- 172.21.15.53:0/3971465306 wait complete. 2023-11-28T11:19:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.142+0000 7f95e2bff700 1 Processor -- start 2023-11-28T11:19:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.142+0000 7f95e2bff700 1 -- start start 2023-11-28T11:19:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15c9b0 0x7f95dc0b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc161340 0x7f95dc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc150bb0 con 0x7f95dc0d2320 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0c7d60 con 0x7f95dc15c9b0 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e2bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0c9240 con 0x7f95dc161340 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e099b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e099b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59418/0 (socket says 172.21.15.53:59418) 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e099b700 1 -- 172.21.15.53:0/2322897963 learned_addr learned my addr 172.21.15.53:0/2322897963 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95e119c700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc161340 0x7f95dc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.143+0000 7f95dbfff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15c9b0 0x7f95dc0b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 -- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc161340 msgr2=0x7f95dc0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc161340 0x7f95dc0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 -- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15c9b0 msgr2=0x7f95dc0b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15c9b0 0x7f95dc0b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 -- 172.21.15.53:0/2322897963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80094a0 con 0x7f95dc0d2320 2023-11-28T11:19:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e099b700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e810 tx=0x7f95d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c070 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0013610 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d00139c0 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc0b5130 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.144+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0d15e0 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.145+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc0b52c0 con 0x7f95dc0d2320 2023-11-28T11:19:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.147+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f95d0023070 con 0x7f95dc0d2320 2023-11-28T11:19:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.148+0000 7f95d97fa700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 0x7f95c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.148+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f95d0085950 con 0x7f95dc0d2320 2023-11-28T11:19:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.148+0000 7f95dbfff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 0x7f95c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.149+0000 7f95dbfff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 0x7f95c006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95c80090b0 tx=0x7f95c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.152+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95d0052190 con 0x7f95dc0d2320 2023-11-28T11:19:44.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.399+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f95dc0c7670 con 0x7f95dc0d2320 2023-11-28T11:19:44.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.406+0000 7f95d97fa700 1 -- 172.21.15.53:0/2322897963 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f95d000f040 con 0x7f95dc0d2320 2023-11-28T11:19:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.411+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 msgr2=0x7f95c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.411+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 0x7f95c006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95c80090b0 tx=0x7f95c8008030 comp rx=0 tx=0).stop 2023-11-28T11:19:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.411+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 msgr2=0x7f95dc0c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.411+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e810 tx=0x7f95d000eb20 comp rx=0 tx=0).stop 2023-11-28T11:19:44.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 shutdown_connections 2023-11-28T11:19:44.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95c00682f0 0x7f95c006a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc161340 0x7f95dc0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95dc0d2320 0x7f95dc0c7820 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 --2- 172.21.15.53:0/2322897963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15c9b0 0x7f95dc0b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 >> 172.21.15.53:0/2322897963 conn(0x7f95dc072600 msgr2=0x7f95dc15a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.413+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 shutdown_connections 2023-11-28T11:19:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.414+0000 7f95e2bff700 1 -- 172.21.15.53:0/2322897963 wait complete. 2023-11-28T11:19:44.423 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T11:19:44.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:44.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.654+0000 7f37265ff700 1 Processor -- start 2023-11-28T11:19:44.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.654+0000 7f37265ff700 1 -- start start 2023-11-28T11:19:44.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f37265ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f37265ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 0x7f37200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f37265ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37200d1950 con 0x7f37200d2320 2023-11-28T11:19:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f37265ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200b5800 con 0x7f3720161340 2023-11-28T11:19:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f37265ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200b5520 con 0x7f372015c9b0 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f371ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f371f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 0x7f37200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f3724b9c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f37200d2320 0x7f37200cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:55960/0 (socket says 172.21.15.53:55960) 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f371ffff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:40698/0 (socket says 172.21.15.53:40698) 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.656+0000 7f3724b9c700 1 -- 172.21.15.53:0/3591714016 learned_addr learned my addr 172.21.15.53:0/3591714016 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:44.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371effd700 1 -- 172.21.15.53:0/3591714016 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981946721 0 0) 0x7f37200d1950 con 0x7f37200d2320 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371effd700 1 -- 172.21.15.53:0/3591714016 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37080035b0 con 0x7f37200d2320 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371ffff700 1 -- 172.21.15.53:0/3591714016 >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 msgr2=0x7f37200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371ffff700 1 --2- 172.21.15.53:0/3591714016 >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 0x7f37200d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371ffff700 1 -- 172.21.15.53:0/3591714016 >> v1:172.21.15.157:6789/0 conn(0x7f37200d2320 legacy=0x7f37200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371ffff700 1 -- 172.21.15.53:0/3591714016 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200d1300 con 0x7f372015c9b0 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371f7fe700 1 --2- 172.21.15.53:0/3591714016 >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 0x7f37200d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371ffff700 1 --2- 172.21.15.53:0/3591714016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3710009840 tx=0x7f3710009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d7542b7452edf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371effd700 1 -- 172.21.15.53:0/3591714016 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011830 con 0x7f372015c9b0 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371effd700 1 -- 172.21.15.53:0/3591714016 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710011990 con 0x7f372015c9b0 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.657+0000 7f371effd700 1 -- 172.21.15.53:0/3591714016 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011d20 con 0x7f372015c9b0 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 -- 172.21.15.53:0/3591714016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 msgr2=0x7f37200b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 --2- 172.21.15.53:0/3591714016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3710009840 tx=0x7f3710009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 -- 172.21.15.53:0/3591714016 shutdown_connections 2023-11-28T11:19:44.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 --2- 172.21.15.53:0/3591714016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f37200b4bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 --2- 172.21.15.53:0/3591714016 >> v2:172.21.15.157:3300/0 conn(0x7f3720161340 0x7f37200d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 -- 172.21.15.53:0/3591714016 >> 172.21.15.53:0/3591714016 conn(0x7f3720072600 msgr2=0x7f37200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:44.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 -- 172.21.15.53:0/3591714016 shutdown_connections 2023-11-28T11:19:44.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.658+0000 7f37265ff700 1 -- 172.21.15.53:0/3591714016 wait complete. 2023-11-28T11:19:44.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.660+0000 7f37265ff700 1 Processor -- start 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.660+0000 7f37265ff700 1 -- start start 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f3720156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 0x7f3720150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720151400 con 0x7f37200d2320 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37201547d0 con 0x7f3720161340 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f37265ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37201544f0 con 0x7f372015c9b0 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f371ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f371ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59430/0 (socket says 172.21.15.53:59430) 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f371ffff700 1 -- 172.21.15.53:0/54290034 learned_addr learned my addr 172.21.15.53:0/54290034 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f371f7fe700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f3720156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.661+0000 7f3724b9c700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 0x7f3720150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 -- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 msgr2=0x7f3720156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f3720156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 -- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 msgr2=0x7f3720150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 0x7f3720150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 -- 172.21.15.53:0/54290034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140094a0 con 0x7f37200d2320 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f3724b9c700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 0x7f3720150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371ffff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37100090b0 tx=0x7f3710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100036a0 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371001f070 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37100094a0 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710003a60 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.662+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37200b5130 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.663+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37200d15e0 con 0x7f37200d2320 2023-11-28T11:19:44.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.665+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f371000f050 con 0x7f37200d2320 2023-11-28T11:19:44.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.665+0000 7f371cff9700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 0x7f370c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:44.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.665+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3710021070 con 0x7f37200d2320 2023-11-28T11:19:44.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.666+0000 7f371f7fe700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 0x7f370c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:44.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.667+0000 7f371f7fe700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 0x7f370c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3720158e70 tx=0x7f3714009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:44.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.670+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3710053d00 con 0x7f37200d2320 2023-11-28T11:19:44.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:44.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:44.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.876+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37200b52c0 con 0x7f37200d2320 2023-11-28T11:19:44.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.878+0000 7f371cff9700 1 -- 172.21.15.53:0/54290034 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17992 (secure 0 0 0) 0x7f3710057320 con 0x7f37200d2320 2023-11-28T11:19:44.879 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:44.879 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":47,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:44.400782+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 msgr2=0x7f370c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 0x7f370c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3720158e70 tx=0x7f3714009210 comp rx=0 tx=0).stop 2023-11-28T11:19:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 msgr2=0x7f37201557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37100090b0 tx=0x7f3710004750 comp rx=0 tx=0).stop 2023-11-28T11:19:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 shutdown_connections 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f370c0682f0 0x7f370c06a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720161340 0x7f3720150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37200d2320 0x7f37201557a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 --2- 172.21.15.53:0/54290034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f372015c9b0 0x7f3720156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 >> 172.21.15.53:0/54290034 conn(0x7f3720072600 msgr2=0x7f372015fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 shutdown_connections 2023-11-28T11:19:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:44.880+0000 7f37265ff700 1 -- 172.21.15.53:0/54290034 wait complete. 2023-11-28T11:19:44.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.142+0000 7f670d1ff700 1 Processor -- start 2023-11-28T11:19:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.142+0000 7f670d1ff700 1 -- start start 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 0x7f67080b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 0x7f67080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67080d2320 0x7f67080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080b5800 con 0x7f67080d2320 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080d1950 con 0x7f6708161340 2023-11-28T11:19:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670d1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b5520 con 0x7f670815c9b0 2023-11-28T11:19:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670759e700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67080d2320 0x7f67080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670759e700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67080d2320 0x7f67080cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59444/0 (socket says 172.21.15.53:59444) 2023-11-28T11:19:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.143+0000 7f670759e700 1 -- 172.21.15.53:0/1151277130 learned_addr learned my addr 172.21.15.53:0/1151277130 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670659c700 1 --2- 172.21.15.53:0/1151277130 >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 0x7f67080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 -- 172.21.15.53:0/1151277130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 msgr2=0x7f67080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f6706d9d700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 0x7f67080b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 0x7f67080b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 -- 172.21.15.53:0/1151277130 >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 msgr2=0x7f67080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 --2- 172.21.15.53:0/1151277130 >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 0x7f67080d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 -- 172.21.15.53:0/1151277130 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080d1300 con 0x7f67080d2320 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670659c700 1 --2- 172.21.15.53:0/1151277130 >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 0x7f67080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f6706d9d700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 0x7f67080b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f670759e700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f67080cd7c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f670806bc00 tx=0x7f670000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a1787c79727ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f6705d9b700 1 -- 172.21.15.53:0/1151277130 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000e070 con 0x7f67080d2320 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f6705d9b700 1 -- 172.21.15.53:0/1151277130 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000b9c0 con 0x7f67080d2320 2023-11-28T11:19:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.144+0000 7f6705d9b700 1 -- 172.21.15.53:0/1151277130 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000bd70 con 0x7f67080d2320 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 -- 172.21.15.53:0/1151277130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 msgr2=0x7f67080cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f67080cd7c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f670806bc00 tx=0x7f670000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 -- 172.21.15.53:0/1151277130 shutdown_connections 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f670815c9b0 0x7f67080b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 --2- 172.21.15.53:0/1151277130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f67080cd7c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 --2- 172.21.15.53:0/1151277130 >> v2:172.21.15.157:3300/0 conn(0x7f6708161340 0x7f67080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 -- 172.21.15.53:0/1151277130 >> 172.21.15.53:0/1151277130 conn(0x7f6708072600 msgr2=0x7f67080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 -- 172.21.15.53:0/1151277130 shutdown_connections 2023-11-28T11:19:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.145+0000 7f670d1ff700 1 -- 172.21.15.53:0/1151277130 wait complete. 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.147+0000 7f670d1ff700 1 Processor -- start 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.147+0000 7f670d1ff700 1 -- start start 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670815c9b0 0x7f6708159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6708161340 0x7f6708159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708151570 con 0x7f67080d2320 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708151780 con 0x7f670815c9b0 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670d1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6708151a60 con 0x7f6708161340 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59450/0 (socket says 172.21.15.53:59450) 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 -- 172.21.15.53:0/2514593069 learned_addr learned my addr 172.21.15.53:0/2514593069 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670659c700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670815c9b0 0x7f6708159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f670759e700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6708161340 0x7f6708159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 -- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6708161340 msgr2=0x7f6708159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6708161340 0x7f6708159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 -- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670815c9b0 msgr2=0x7f6708159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670815c9b0 0x7f6708159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.148+0000 7f6706d9d700 1 -- 172.21.15.53:0/2514593069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009b30 con 0x7f67080d2320 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f6706d9d700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c7f0 tx=0x7f66f800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011800 con 0x7f67080d2320 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8011960 con 0x7f67080d2320 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011cf0 con 0x7f67080d2320 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67080b5130 con 0x7f67080d2320 2023-11-28T11:19:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.149+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67080d15e0 con 0x7f67080d2320 2023-11-28T11:19:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.150+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67080b52c0 con 0x7f67080d2320 2023-11-28T11:19:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.151+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66f8021070 con 0x7f67080d2320 2023-11-28T11:19:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.152+0000 7f66ef7fe700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 0x7f66f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.152+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66f8085bb0 con 0x7f67080d2320 2023-11-28T11:19:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.152+0000 7f670659c700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 0x7f66f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.153+0000 7f670659c700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 0x7f66f406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f670815fce0 tx=0x7f66fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.156+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66f80522a0 con 0x7f67080d2320 2023-11-28T11:19:45.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.360+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67080745c0 con 0x7f67080d2320 2023-11-28T11:19:45.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.361+0000 7f66ef7fe700 1 -- 172.21.15.53:0/2514593069 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17992 (secure 0 0 0) 0x7f66f801f020 con 0x7f67080d2320 2023-11-28T11:19:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:45.362 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":47,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:44.400782+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:45.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 msgr2=0x7f66f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 0x7f66f406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f670815fce0 tx=0x7f66fc008040 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 msgr2=0x7f6708150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c7f0 tx=0x7f66f800cb00 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 shutdown_connections 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f66f40682f0 0x7f66f406a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6708161340 0x7f6708159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67080d2320 0x7f6708150750 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 --2- 172.21.15.53:0/2514593069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670815c9b0 0x7f6708159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 >> 172.21.15.53:0/2514593069 conn(0x7f6708072600 msgr2=0x7f670815cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.363+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 shutdown_connections 2023-11-28T11:19:45.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.364+0000 7f670d1ff700 1 -- 172.21.15.53:0/2514593069 wait complete. 2023-11-28T11:19:45.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:19:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.588+0000 7f719f1ff700 1 Processor -- start 2023-11-28T11:19:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.588+0000 7f719f1ff700 1 -- start start 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.589+0000 7f719f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7198161340 0x7f71980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.589+0000 7f719f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71980d2320 0x7f71980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.589+0000 7f719f1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71980d1950 con 0x7f719815c9b0 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.589+0000 7f719f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71980b5800 con 0x7f71980d2320 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.589+0000 7f719f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71980b5300 con 0x7f7198161340 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71980d2320 0x7f71980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71980d2320 0x7f71980cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59452/0 (socket says 172.21.15.53:59452) 2023-11-28T11:19:45.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 -- 172.21.15.53:0/456021065 learned_addr learned my addr 172.21.15.53:0/456021065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f7197fff700 1 --2- 172.21.15.53:0/456021065 >> v2:172.21.15.157:3300/0 conn(0x7f7198161340 0x7f71980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 -- 172.21.15.53:0/456021065 >> v2:172.21.15.157:3300/0 conn(0x7f7198161340 msgr2=0x7f71980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 --2- 172.21.15.53:0/456021065 >> v2:172.21.15.157:3300/0 conn(0x7f7198161340 0x7f71980d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 -- 172.21.15.53:0/456021065 >> v1:172.21.15.157:6789/0 conn(0x7f719815c9b0 legacy=0x7f71980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 -- 172.21.15.53:0/456021065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980b5520 con 0x7f71980d2320 2023-11-28T11:19:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f719d79c700 1 --2- 172.21.15.53:0/456021065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71980d2320 0x7f71980cd7c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71980685c0 tx=0x7f719000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f6424aa91c9558 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.590+0000 7f71977fe700 1 -- 172.21.15.53:0/456021065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000e070 con 0x7f71980d2320 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f71977fe700 1 -- 172.21.15.53:0/456021065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719000b980 con 0x7f71980d2320 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f71977fe700 1 -- 172.21.15.53:0/456021065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000bd30 con 0x7f71980d2320 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 -- 172.21.15.53:0/456021065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71980d2320 msgr2=0x7f71980cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 --2- 172.21.15.53:0/456021065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71980d2320 0x7f71980cd7c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71980685c0 tx=0x7f719000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 -- 172.21.15.53:0/456021065 shutdown_connections 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 --2- 172.21.15.53:0/456021065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71980d2320 0x7f71980cd7c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 --2- 172.21.15.53:0/456021065 >> v2:172.21.15.157:3300/0 conn(0x7f7198161340 0x7f71980d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 -- 172.21.15.53:0/456021065 >> 172.21.15.53:0/456021065 conn(0x7f7198072600 msgr2=0x7f71980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 -- 172.21.15.53:0/456021065 shutdown_connections 2023-11-28T11:19:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.591+0000 7f719f1ff700 1 -- 172.21.15.53:0/456021065 wait complete. 2023-11-28T11:19:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.593+0000 7f719f1ff700 1 Processor -- start 2023-11-28T11:19:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.593+0000 7f719f1ff700 1 -- start start 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 0x7f71980c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 0x7f71980b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71981521e0 con 0x7f719815c9b0 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7198154dd0 con 0x7f71980d2320 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f719f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7198150c90 con 0x7f7198161340 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f7197fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f7197fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59456/0 (socket says 172.21.15.53:59456) 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.594+0000 7f7197fff700 1 -- 172.21.15.53:0/410156436 learned_addr learned my addr 172.21.15.53:0/410156436 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 -- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 msgr2=0x7f71980b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719d79c700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 0x7f71980b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719cf9b700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 0x7f71980c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 0x7f71980b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 -- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 msgr2=0x7f71980c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 0x7f71980c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 -- 172.21.15.53:0/410156436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009b30 con 0x7f719815c9b0 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719d79c700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 0x7f71980b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719cf9b700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 0x7f71980c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f7197fff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f718400e810 tx=0x7f718400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184013610 con 0x7f719815c9b0 2023-11-28T11:19:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7184013770 con 0x7f719815c9b0 2023-11-28T11:19:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184013b20 con 0x7f719815c9b0 2023-11-28T11:19:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71980d15e0 con 0x7f719815c9b0 2023-11-28T11:19:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.595+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71980d1300 con 0x7f719815c9b0 2023-11-28T11:19:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.596+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719815cd80 con 0x7f719815c9b0 2023-11-28T11:19:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.598+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7184023070 con 0x7f719815c9b0 2023-11-28T11:19:45.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.599+0000 7f71957fa700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 0x7f717c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:45.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.599+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7184085ab0 con 0x7f719815c9b0 2023-11-28T11:19:45.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.599+0000 7f719cf9b700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 0x7f717c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:45.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.600+0000 7f719cf9b700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 0x7f717c06a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f718c001670 tx=0x7f718c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.603+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71840521a0 con 0x7f719815c9b0 2023-11-28T11:19:45.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:45.852+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f71980d08a0 con 0x7f719815c9b0 2023-11-28T11:19:46.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.411+0000 7f71957fa700 1 -- 172.21.15.53:0/410156436 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f71840557c0 con 0x7f719815c9b0 2023-11-28T11:19:46.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 msgr2=0x7f717c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:46.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 0x7f717c06a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f718c001670 tx=0x7f718c009040 comp rx=0 tx=0).stop 2023-11-28T11:19:46.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 msgr2=0x7f7198150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:46.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f718400e810 tx=0x7f718400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 shutdown_connections 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f717c0682f0 0x7f717c06a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7198161340 0x7f71980b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71980d2320 0x7f71980c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 --2- 172.21.15.53:0/410156436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719815c9b0 0x7f7198150750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 >> 172.21.15.53:0/410156436 conn(0x7f7198072600 msgr2=0x7f719815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 shutdown_connections 2023-11-28T11:19:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.414+0000 7f719f1ff700 1 -- 172.21.15.53:0/410156436 wait complete. 2023-11-28T11:19:46.438 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:19:46.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:19:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.645+0000 7f2cc47ff700 1 Processor -- start 2023-11-28T11:19:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.645+0000 7f2cc47ff700 1 -- start start 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c9b0 0x7f2cbc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cbc161340 0x7f2cbc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cbc0d2320 0x7f2cbc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b5800 con 0x7f2cbc161340 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0d1950 con 0x7f2cbc0d2320 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc47ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b5520 con 0x7f2cbc15c9b0 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc1d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cbc161340 0x7f2cbc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc1d9a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cbc161340 0x7f2cbc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59458/0 (socket says 172.21.15.53:59458) 2023-11-28T11:19:46.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc1d9a700 1 -- 172.21.15.53:0/2992912356 learned_addr learned my addr 172.21.15.53:0/2992912356 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc259b700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c9b0 0x7f2cbc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.646+0000 7f2cc2d9c700 1 --2- 172.21.15.53:0/2992912356 >> v2:172.21.15.157:3300/0 conn(0x7f2cbc0d2320 0x7f2cbc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 -- 172.21.15.53:0/2992912356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c9b0 msgr2=0x7f2cbc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c9b0 0x7f2cbc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 -- 172.21.15.53:0/2992912356 >> v2:172.21.15.157:3300/0 conn(0x7f2cbc0d2320 msgr2=0x7f2cbc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/2992912356 >> v2:172.21.15.157:3300/0 conn(0x7f2cbc0d2320 0x7f2cbc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 -- 172.21.15.53:0/2992912356 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0d1300 con 0x7f2cbc161340 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc161340 0x7f2cbc0d1de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc08c4d0 tx=0x7f2cac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c2839cbb9420ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1599700 1 -- 172.21.15.53:0/2992912356 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00b870 con 0x7f2cbc161340 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1599700 1 -- 172.21.15.53:0/2992912356 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00b9d0 con 0x7f2cbc161340 2023-11-28T11:19:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.647+0000 7f2cc1599700 1 -- 172.21.15.53:0/2992912356 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00bd60 con 0x7f2cbc161340 2023-11-28T11:19:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 -- 172.21.15.53:0/2992912356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc161340 msgr2=0x7f2cbc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc161340 0x7f2cbc0d1de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc08c4d0 tx=0x7f2cac009bd0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 -- 172.21.15.53:0/2992912356 shutdown_connections 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c9b0 0x7f2cbc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/2992912356 >> v2:172.21.15.157:3300/0 conn(0x7f2cbc0d2320 0x7f2cbc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/2992912356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc161340 0x7f2cbc0d1de0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 -- 172.21.15.53:0/2992912356 >> 172.21.15.53:0/2992912356 conn(0x7f2cbc072600 msgr2=0x7f2cbc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 -- 172.21.15.53:0/2992912356 shutdown_connections 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.648+0000 7f2cc47ff700 1 -- 172.21.15.53:0/2992912356 wait complete. 2023-11-28T11:19:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.650+0000 7f2cc47ff700 1 Processor -- start 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.650+0000 7f2cc47ff700 1 -- start start 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 0x7f2cbc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc161340 0x7f2cbc15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c7bf0 con 0x7f2cbc0d2320 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0cd360 con 0x7f2cbc15c9b0 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc47ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0cd640 con 0x7f2cbc161340 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59466/0 (socket says 172.21.15.53:59466) 2023-11-28T11:19:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 -- 172.21.15.53:0/267501338 learned_addr learned my addr 172.21.15.53:0/267501338 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 0x7f2cbc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc2d9c700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc161340 0x7f2cbc15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 -- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc161340 msgr2=0x7f2cbc15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc161340 0x7f2cbc15ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.651+0000 7f2cc259b700 1 -- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 msgr2=0x7f2cbc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc259b700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 0x7f2cbc0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc259b700 1 -- 172.21.15.53:0/267501338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009550 con 0x7f2cbc0d2320 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 0x7f2cbc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc259b700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d800 tx=0x7f2cb800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80125f0 con 0x7f2cbc0d2320 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb8012750 con 0x7f2cbc0d2320 2023-11-28T11:19:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8012b00 con 0x7f2cbc0d2320 2023-11-28T11:19:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0b5130 con 0x7f2cbc0d2320 2023-11-28T11:19:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.652+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc0d15e0 con 0x7f2cbc0d2320 2023-11-28T11:19:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.653+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc0b52c0 con 0x7f2cbc0d2320 2023-11-28T11:19:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.654+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2cb8012c60 con 0x7f2cbc0d2320 2023-11-28T11:19:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.655+0000 7f2cb2ffd700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 0x7f2ca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.655+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cb8086220 con 0x7f2cbc0d2320 2023-11-28T11:19:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.655+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 0x7f2ca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.656+0000 7f2cc1d9a700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 0x7f2ca006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:46.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.659+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2cb8052990 con 0x7f2cbc0d2320 2023-11-28T11:19:46.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:46.908+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2cbc0745c0 con 0x7f2cbc0d2320 2023-11-28T11:19:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.417+0000 7f2cb2ffd700 1 -- 172.21.15.53:0/267501338 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2cb8055fb0 con 0x7f2cbc0d2320 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 msgr2=0x7f2ca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 0x7f2ca006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac008030 comp rx=0 tx=0).stop 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 msgr2=0x7f2cbc0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d800 tx=0x7f2cb800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 shutdown_connections 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ca00682f0 0x7f2ca006a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc161340 0x7f2cbc15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cbc0d2320 0x7f2cbc0ccc80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 --2- 172.21.15.53:0/267501338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15c9b0 0x7f2cbc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 >> 172.21.15.53:0/267501338 conn(0x7f2cbc072600 msgr2=0x7f2cbc15a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:47.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 shutdown_connections 2023-11-28T11:19:47.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.420+0000 7f2cc47ff700 1 -- 172.21.15.53:0/267501338 wait complete. 2023-11-28T11:19:47.440 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:19:47.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:47.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.641+0000 7ff4d01ff700 1 Processor -- start 2023-11-28T11:19:47.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.641+0000 7ff4d01ff700 1 -- start start 2023-11-28T11:19:47.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 0x7ff4c80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 0x7ff4c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4c80d2320 0x7ff4c80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80b5800 con 0x7ff4c80d2320 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80d1950 con 0x7ff4c815c9b0 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.642+0000 7ff4d01ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c80b5520 con 0x7ff4c8161340 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4c80d2320 0x7ff4c80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4c80d2320 0x7ff4c80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59478/0 (socket says 172.21.15.53:59478) 2023-11-28T11:19:47.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 -- 172.21.15.53:0/2685394572 learned_addr learned my addr 172.21.15.53:0/2685394572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4cd79a700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 0x7ff4c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 -- 172.21.15.53:0/2685394572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 msgr2=0x7ff4c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4cdf9b700 1 --2- 172.21.15.53:0/2685394572 >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 0x7ff4c80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 0x7ff4c80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 -- 172.21.15.53:0/2685394572 >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 msgr2=0x7ff4c80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/2685394572 >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 0x7ff4c80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 -- 172.21.15.53:0/2685394572 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80d1300 con 0x7ff4c80d2320 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4cd79a700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 0x7ff4c80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4cdf9b700 1 --2- 172.21.15.53:0/2685394572 >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 0x7ff4c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.643+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c80d2320 0x7ff4c80cd7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8067460 tx=0x7ff4c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2fa2b3bc078192 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4ccf99700 1 -- 172.21.15.53:0/2685394572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000e070 con 0x7ff4c80d2320 2023-11-28T11:19:47.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4ccf99700 1 -- 172.21.15.53:0/2685394572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c000b9c0 con 0x7ff4c80d2320 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4ccf99700 1 -- 172.21.15.53:0/2685394572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000bd70 con 0x7ff4c80d2320 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 -- 172.21.15.53:0/2685394572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c80d2320 msgr2=0x7ff4c80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c80d2320 0x7ff4c80cd7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8067460 tx=0x7ff4c000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 -- 172.21.15.53:0/2685394572 shutdown_connections 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c80d2320 0x7ff4c80cd7c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/2685394572 >> v2:172.21.15.157:3300/0 conn(0x7ff4c815c9b0 0x7ff4c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/2685394572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c8161340 0x7ff4c80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 -- 172.21.15.53:0/2685394572 >> 172.21.15.53:0/2685394572 conn(0x7ff4c8072600 msgr2=0x7ff4c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 -- 172.21.15.53:0/2685394572 shutdown_connections 2023-11-28T11:19:47.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.644+0000 7ff4d01ff700 1 -- 172.21.15.53:0/2685394572 wait complete. 2023-11-28T11:19:47.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.646+0000 7ff4d01ff700 1 Processor -- start 2023-11-28T11:19:47.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.646+0000 7ff4d01ff700 1 -- start start 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80d2320 0x7ff4c815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c815c9b0 0x7ff4c8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c815eae0 con 0x7ff4c8161340 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8151ab0 con 0x7ff4c815c9b0 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4d01ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8152840 con 0x7ff4c80d2320 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.647+0000 7ff4ce79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59488/0 (socket says 172.21.15.53:59488) 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 -- 172.21.15.53:0/4144890403 learned_addr learned my addr 172.21.15.53:0/4144890403 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4cdf9b700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80d2320 0x7ff4c815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4cd79a700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c815c9b0 0x7ff4c8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 -- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80d2320 msgr2=0x7ff4c815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80d2320 0x7ff4c815e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 -- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c815c9b0 msgr2=0x7ff4c8152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c815c9b0 0x7ff4c8152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 -- 172.21.15.53:0/4144890403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0009b30 con 0x7ff4c8161340 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4ce79c700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b370 tx=0x7ff4c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0004310 con 0x7ff4c8161340 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c0004470 con 0x7ff4c8161340 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.648+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c80b5130 con 0x7ff4c8161340 2023-11-28T11:19:47.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.649+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0004820 con 0x7ff4c8161340 2023-11-28T11:19:47.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.649+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c80d15e0 con 0x7ff4c8161340 2023-11-28T11:19:47.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.650+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c80b52c0 con 0x7ff4c8161340 2023-11-28T11:19:47.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.651+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff4c001b430 con 0x7ff4c8161340 2023-11-28T11:19:47.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.652+0000 7ff4be7fc700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 0x7ff4ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:47.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.652+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4c0087090 con 0x7ff4c8161340 2023-11-28T11:19:47.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.652+0000 7ff4cdf9b700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 0x7ff4ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:47.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.653+0000 7ff4cdf9b700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 0x7ff4ac06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006fd0 tx=0x7ff4c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:47.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.656+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff4c0053890 con 0x7ff4c8161340 2023-11-28T11:19:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.905+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4c80745c0 con 0x7ff4c8161340 2023-11-28T11:19:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.906+0000 7ff4be7fc700 1 -- 172.21.15.53:0/4144890403 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ff4c0056eb0 con 0x7ff4c8161340 2023-11-28T11:19:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:47.907 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:19:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 msgr2=0x7ff4ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 0x7ff4ac06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006fd0 tx=0x7ff4c4008040 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 msgr2=0x7ff4c81f1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b370 tx=0x7ff4c0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 shutdown_connections 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff4ac0682f0 0x7ff4ac06a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c80d2320 0x7ff4c815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4c8161340 0x7ff4c81f1b40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 --2- 172.21.15.53:0/4144890403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c815c9b0 0x7ff4c8152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 >> 172.21.15.53:0/4144890403 conn(0x7ff4c8072600 msgr2=0x7ff4c815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 shutdown_connections 2023-11-28T11:19:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:47.908+0000 7ff4d01ff700 1 -- 172.21.15.53:0/4144890403 wait complete. 2023-11-28T11:19:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:19:47.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:19:48.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.128+0000 7f65e15ff700 1 Processor -- start 2023-11-28T11:19:48.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.129+0000 7f65e15ff700 1 -- start start 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65e15ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65dc161340 0x7f65dc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65e15ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0d1950 con 0x7f65dc15c9b0 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65e15ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0b5520 con 0x7f65dc0d2320 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65e15ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0b5800 con 0x7f65dc161340 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65da7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65dc161340 0x7f65dc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65da7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65dc161340 0x7f65dc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59504/0 (socket says 172.21.15.53:59504) 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.130+0000 7f65da7fc700 1 -- 172.21.15.53:0/3001777021 learned_addr learned my addr 172.21.15.53:0/3001777021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65da7fc700 1 -- 172.21.15.53:0/3001777021 >> v1:172.21.15.157:6789/0 conn(0x7f65dc0d2320 legacy=0x7f65dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65da7fc700 1 -- 172.21.15.53:0/3001777021 >> v1:172.21.15.53:6789/0 conn(0x7f65dc15c9b0 legacy=0x7f65dc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65da7fc700 1 -- 172.21.15.53:0/3001777021 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0d1300 con 0x7f65dc161340 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65da7fc700 1 --2- 172.21.15.53:0/3001777021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc161340 0x7f65dc0d1de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65d00098f0 tx=0x7f65d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cafd80166985df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65d9ffb700 1 -- 172.21.15.53:0/3001777021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0004230 con 0x7f65dc161340 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65d9ffb700 1 -- 172.21.15.53:0/3001777021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0004390 con 0x7f65dc161340 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.131+0000 7f65d9ffb700 1 -- 172.21.15.53:0/3001777021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00036a0 con 0x7f65dc161340 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 -- 172.21.15.53:0/3001777021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc161340 msgr2=0x7f65dc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 --2- 172.21.15.53:0/3001777021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc161340 0x7f65dc0d1de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65d00098f0 tx=0x7f65d0009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 -- 172.21.15.53:0/3001777021 shutdown_connections 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 --2- 172.21.15.53:0/3001777021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc161340 0x7f65dc0d1de0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 -- 172.21.15.53:0/3001777021 >> 172.21.15.53:0/3001777021 conn(0x7f65dc072600 msgr2=0x7f65dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 -- 172.21.15.53:0/3001777021 shutdown_connections 2023-11-28T11:19:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.132+0000 7f65e15ff700 1 -- 172.21.15.53:0/3001777021 wait complete. 2023-11-28T11:19:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.134+0000 7f65e15ff700 1 Processor -- start 2023-11-28T11:19:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.134+0000 7f65e15ff700 1 -- start start 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 0x7f65dc1e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 0x7f65dc156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c6630 con 0x7f65dc0d2320 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1e81d0 con 0x7f65dc15c9b0 2023-11-28T11:19:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65e15ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c7a30 con 0x7f65dc161340 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59506/0 (socket says 172.21.15.53:59506) 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 -- 172.21.15.53:0/1286646315 learned_addr learned my addr 172.21.15.53:0/1286646315 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65db7fe700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 0x7f65dc156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 -- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 msgr2=0x7f65dc156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65da7fc700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 0x7f65dc1e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 0x7f65dc156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 -- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 msgr2=0x7f65dc1e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 0x7f65dc1e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65daffd700 1 -- 172.21.15.53:0/1286646315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0009550 con 0x7f65dc0d2320 2023-11-28T11:19:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65db7fe700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 0x7f65dc156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.135+0000 7f65da7fc700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 0x7f65dc1e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65daffd700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c4c0 tx=0x7f65cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0052d0 con 0x7f65dc0d2320 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc005450 con 0x7f65dc0d2320 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc006aa0 con 0x7f65dc0d2320 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc15cd80 con 0x7f65dc0d2320 2023-11-28T11:19:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.136+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc0d15e0 con 0x7f65dc0d2320 2023-11-28T11:19:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.137+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc15cf10 con 0x7f65dc0d2320 2023-11-28T11:19:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.138+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65cc010050 con 0x7f65dc0d2320 2023-11-28T11:19:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.139+0000 7f65c37fe700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 0x7f65c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.139+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65cc084ff0 con 0x7f65dc0d2320 2023-11-28T11:19:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.139+0000 7f65da7fc700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 0x7f65c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.140+0000 7f65da7fc700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 0x7f65c806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65dc15da10 tx=0x7f65d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.143+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65cc04a5e0 con 0x7f65dc0d2320 2023-11-28T11:19:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.347+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65dc066d60 con 0x7f65dc0d2320 2023-11-28T11:19:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.348+0000 7f65c37fe700 1 -- 172.21.15.53:0/1286646315 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f65cc051760 con 0x7f65dc0d2320 2023-11-28T11:19:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:48.350 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:45.403641+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 msgr2=0x7f65c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 0x7f65c806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65dc15da10 tx=0x7f65d0008030 comp rx=0 tx=0).stop 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 msgr2=0x7f65dc0c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c4c0 tx=0x7f65cc00c880 comp rx=0 tx=0).stop 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 shutdown_connections 2023-11-28T11:19:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f65c80682f0 0x7f65c806a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65dc161340 0x7f65dc156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0d2320 0x7f65dc0c60f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 --2- 172.21.15.53:0/1286646315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65dc15c9b0 0x7f65dc1e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 >> 172.21.15.53:0/1286646315 conn(0x7f65dc072600 msgr2=0x7f65dc0b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 shutdown_connections 2023-11-28T11:19:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.351+0000 7f65e15ff700 1 -- 172.21.15.53:0/1286646315 wait complete. 2023-11-28T11:19:48.373 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:19:48.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:19:48.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.575+0000 7f80ac7ff700 1 Processor -- start 2023-11-28T11:19:48.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.576+0000 7f80ac7ff700 1 -- start start 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80ac7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80a415c9b0 0x7f80a40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80ac7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80a4161340 0x7f80a40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80ac7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40d1950 con 0x7f80a40d2320 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80ac7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40b5800 con 0x7f80a415c9b0 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80ac7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40b5520 con 0x7f80a4161340 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80a415c9b0 0x7f80a40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aad9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f80a40d2320 0x7f80a40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56166/0 (socket says 172.21.15.53:56166) 2023-11-28T11:19:48.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aad9c700 1 -- 172.21.15.53:0/1593916701 learned_addr learned my addr 172.21.15.53:0/1593916701 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:48.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1593916701 >> v2:172.21.15.53:3300/0 conn(0x7f80a415c9b0 0x7f80a40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59510/0 (socket says 172.21.15.53:59510) 2023-11-28T11:19:48.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80a9d9a700 1 --2- 172.21.15.53:0/1593916701 >> v2:172.21.15.157:3300/0 conn(0x7f80a4161340 0x7f80a40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 -- 172.21.15.53:0/1593916701 >> v2:172.21.15.157:3300/0 conn(0x7f80a4161340 msgr2=0x7f80a40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1593916701 >> v2:172.21.15.157:3300/0 conn(0x7f80a4161340 0x7f80a40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 -- 172.21.15.53:0/1593916701 >> v1:172.21.15.53:6789/0 conn(0x7f80a40d2320 legacy=0x7f80a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.577+0000 7f80aa59b700 1 -- 172.21.15.53:0/1593916701 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40d1300 con 0x7f80a415c9b0 2023-11-28T11:19:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1593916701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a40b4bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f80a40685c0 tx=0x7f80a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef44db576628f55c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80a9599700 1 -- 172.21.15.53:0/1593916701 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0011830 con 0x7f80a415c9b0 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80a9599700 1 -- 172.21.15.53:0/1593916701 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a0011990 con 0x7f80a415c9b0 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80a9599700 1 -- 172.21.15.53:0/1593916701 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0011d20 con 0x7f80a415c9b0 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1593916701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 msgr2=0x7f80a40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.578+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1593916701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a40b4bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f80a40685c0 tx=0x7f80a0009b20 comp rx=0 tx=0).stop 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1593916701 shutdown_connections 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1593916701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a40b4bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1593916701 >> v2:172.21.15.157:3300/0 conn(0x7f80a4161340 0x7f80a40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1593916701 >> 172.21.15.53:0/1593916701 conn(0x7f80a4072600 msgr2=0x7f80a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1593916701 shutdown_connections 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.579+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1593916701 wait complete. 2023-11-28T11:19:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.581+0000 7f80ac7ff700 1 Processor -- start 2023-11-28T11:19:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.581+0000 7f80ac7ff700 1 -- start start 2023-11-28T11:19:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.582+0000 7f80ac7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a40d2320 0x7f80a41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80ac7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80ac7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a4161340 0x7f80a414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80ac7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41e8650 con 0x7f80a415c9b0 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80ac7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40b7330 con 0x7f80a4161340 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80ac7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41e7d20 con 0x7f80a40d2320 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59518/0 (socket says 172.21.15.53:59518) 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 -- 172.21.15.53:0/1263322226 learned_addr learned my addr 172.21.15.53:0/1263322226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a40d2320 0x7f80a41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80aad9c700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a4161340 0x7f80a414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 -- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a40d2320 msgr2=0x7f80a41505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a40d2320 0x7f80a41505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 -- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a4161340 msgr2=0x7f80a414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a4161340 0x7f80a414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.583+0000 7f80a9d9a700 1 -- 172.21.15.53:0/1263322226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00094a0 con 0x7f80a415c9b0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f80a9d9a700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f809400e810 tx=0x7f809400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400c070 con 0x7f80a415c9b0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094013610 con 0x7f80a415c9b0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940139c0 con 0x7f80a415c9b0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a40b5130 con 0x7f80a415c9b0 2023-11-28T11:19:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.584+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a40d15e0 con 0x7f80a415c9b0 2023-11-28T11:19:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.585+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a40b52c0 con 0x7f80a415c9b0 2023-11-28T11:19:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.586+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8094023070 con 0x7f80a415c9b0 2023-11-28T11:19:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.587+0000 7f809affd700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 0x7f808806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.587+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8094085a60 con 0x7f80a415c9b0 2023-11-28T11:19:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.587+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 0x7f808806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:48.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.588+0000 7f80aa59b700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 0x7f808806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80a00090b0 tx=0x7f80a0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:48.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.591+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8094052240 con 0x7f80a415c9b0 2023-11-28T11:19:48.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.788+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80a40745c0 con 0x7f80880682f0 2023-11-28T11:19:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.798+0000 7f809affd700 1 -- 172.21.15.53:0/1263322226 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311011 (secure 0 0 0) 0x7f80a40745c0 con 0x7f80880682f0 2023-11-28T11:19:48.803 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.802+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 msgr2=0x7f808806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.802+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 0x7f808806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80a00090b0 tx=0x7f80a0008030 comp rx=0 tx=0).stop 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.802+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 msgr2=0x7f80a414d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.802+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f809400e810 tx=0x7f809400eb20 comp rx=0 tx=0).stop 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 shutdown_connections 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f80880682f0 0x7f808806a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a40d2320 0x7f80a41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a4161340 0x7f80a414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 --2- 172.21.15.53:0/1263322226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a415c9b0 0x7f80a414d820 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 >> 172.21.15.53:0/1263322226 conn(0x7f80a4072600 msgr2=0x7f80a415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 shutdown_connections 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:48.803+0000 7f80ac7ff700 1 -- 172.21.15.53:0/1263322226 wait complete. 2023-11-28T11:19:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:19:48.822 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-28T11:19:46.959844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":106724,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":101120,"kb_avail":754867996,"statfs":{"total":773094113280,"available":772984827904,"internally_reserved":0,"allocated":5672960,"data_stored":3080708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":103546880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.077210"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.428394+0000","last_change":"2023-11-28T11:19:44.421978+0000","last_active":"2023-11-28T11:19:44.428394+0000","last_peered":"2023-11-28T11:19:44.428394+0000","last_clean":"2023-11-28T11:19:44.428394+0000","last_became_active":"2023-11-28T11:19:44.421712+0000","last_became_peered":"2023-11-28T11:19:44.421712+0000","last_unstale":"2023-11-28T11:19:44.428394+0000","last_undegraded":"2023-11-28T11:19:44.428394+0000","last_fullsized":"2023-11-28T11:19:44.428394+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:17.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412996+0000","last_change":"2023-11-28T11:19:44.433902+0000","last_active":"2023-11-28T11:19:45.412996+0000","last_peered":"2023-11-28T11:19:45.412996+0000","last_clean":"2023-11-28T11:19:45.412996+0000","last_became_active":"2023-11-28T11:19:44.432689+0000","last_became_peered":"2023-11-28T11:19:44.432689+0000","last_unstale":"2023-11-28T11:19:45.412996+0000","last_undegraded":"2023-11-28T11:19:45.412996+0000","last_fullsized":"2023-11-28T11:19:45.412996+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:40:10.501924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629499+0000","last_change":"2023-11-28T11:19:44.425167+0000","last_active":"2023-11-28T11:19:45.629499+0000","last_peered":"2023-11-28T11:19:45.629499+0000","last_clean":"2023-11-28T11:19:45.629499+0000","last_became_active":"2023-11-28T11:19:44.424531+0000","last_became_peered":"2023-11-28T11:19:44.424531+0000","last_unstale":"2023-11-28T11:19:45.629499+0000","last_undegraded":"2023-11-28T11:19:45.629499+0000","last_fullsized":"2023-11-28T11:19:45.629499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:14.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412704+0000","last_change":"2023-11-28T11:19:44.437722+0000","last_active":"2023-11-28T11:19:45.412704+0000","last_peered":"2023-11-28T11:19:45.412704+0000","last_clean":"2023-11-28T11:19:45.412704+0000","last_became_active":"2023-11-28T11:19:44.437417+0000","last_became_peered":"2023-11-28T11:19:44.437417+0000","last_unstale":"2023-11-28T11:19:45.412704+0000","last_undegraded":"2023-11-28T11:19:45.412704+0000","last_fullsized":"2023-11-28T11:19:45.412704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:31.471220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412521+0000","last_change":"2023-11-28T11:19:44.428328+0000","last_active":"2023-11-28T11:19:45.412521+0000","last_peered":"2023-11-28T11:19:45.412521+0000","last_clean":"2023-11-28T11:19:45.412521+0000","last_became_active":"2023-11-28T11:19:44.428016+0000","last_became_peered":"2023-11-28T11:19:44.428016+0000","last_unstale":"2023-11-28T11:19:45.412521+0000","last_undegraded":"2023-11-28T11:19:45.412521+0000","last_fullsized":"2023-11-28T11:19:45.412521+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:26.387178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.428631+0000","last_change":"2023-11-28T11:19:44.427667+0000","last_active":"2023-11-28T11:19:44.428631+0000","last_peered":"2023-11-28T11:19:44.428631+0000","last_clean":"2023-11-28T11:19:44.428631+0000","last_became_active":"2023-11-28T11:19:44.427446+0000","last_became_peered":"2023-11-28T11:19:44.427446+0000","last_unstale":"2023-11-28T11:19:44.428631+0000","last_undegraded":"2023-11-28T11:19:44.428631+0000","last_fullsized":"2023-11-28T11:19:44.428631+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:25.805157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629092+0000","last_change":"2023-11-28T11:19:44.420659+0000","last_active":"2023-11-28T11:19:45.629092+0000","last_peered":"2023-11-28T11:19:45.629092+0000","last_clean":"2023-11-28T11:19:45.629092+0000","last_became_active":"2023-11-28T11:19:44.420265+0000","last_became_peered":"2023-11-28T11:19:44.420265+0000","last_unstale":"2023-11-28T11:19:45.629092+0000","last_undegraded":"2023-11-28T11:19:45.629092+0000","last_fullsized":"2023-11-28T11:19:45.629092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:39.119998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410896+0000","last_change":"2023-11-28T11:19:44.440246+0000","last_active":"2023-11-28T11:19:45.410896+0000","last_peered":"2023-11-28T11:19:45.410896+0000","last_clean":"2023-11-28T11:19:45.410896+0000","last_became_active":"2023-11-28T11:19:44.439978+0000","last_became_peered":"2023-11-28T11:19:44.439978+0000","last_unstale":"2023-11-28T11:19:45.410896+0000","last_undegraded":"2023-11-28T11:19:45.410896+0000","last_fullsized":"2023-11-28T11:19:45.410896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:30.934789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411377+0000","last_change":"2023-11-28T11:19:44.441037+0000","last_active":"2023-11-28T11:19:45.411377+0000","last_peered":"2023-11-28T11:19:45.411377+0000","last_clean":"2023-11-28T11:19:45.411377+0000","last_became_active":"2023-11-28T11:19:44.440517+0000","last_became_peered":"2023-11-28T11:19:44.440517+0000","last_unstale":"2023-11-28T11:19:45.411377+0000","last_undegraded":"2023-11-28T11:19:45.411377+0000","last_fullsized":"2023-11-28T11:19:45.411377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:51.124532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422371+0000","last_change":"2023-11-28T11:19:44.412751+0000","last_active":"2023-11-28T11:19:44.422371+0000","last_peered":"2023-11-28T11:19:44.422371+0000","last_clean":"2023-11-28T11:19:44.422371+0000","last_became_active":"2023-11-28T11:19:44.412441+0000","last_became_peered":"2023-11-28T11:19:44.412441+0000","last_unstale":"2023-11-28T11:19:44.422371+0000","last_undegraded":"2023-11-28T11:19:44.422371+0000","last_fullsized":"2023-11-28T11:19:44.422371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:07.144404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.426143+0000","last_change":"2023-11-28T11:19:44.422198+0000","last_active":"2023-11-28T11:19:44.426143+0000","last_peered":"2023-11-28T11:19:44.426143+0000","last_clean":"2023-11-28T11:19:44.426143+0000","last_became_active":"2023-11-28T11:19:44.421903+0000","last_became_peered":"2023-11-28T11:19:44.421903+0000","last_unstale":"2023-11-28T11:19:44.426143+0000","last_undegraded":"2023-11-28T11:19:44.426143+0000","last_fullsized":"2023-11-28T11:19:44.426143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:01.437532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411334+0000","last_change":"2023-11-28T11:19:44.434590+0000","last_active":"2023-11-28T11:19:45.411334+0000","last_peered":"2023-11-28T11:19:45.411334+0000","last_clean":"2023-11-28T11:19:45.411334+0000","last_became_active":"2023-11-28T11:19:44.434306+0000","last_became_peered":"2023-11-28T11:19:44.434306+0000","last_unstale":"2023-11-28T11:19:45.411334+0000","last_undegraded":"2023-11-28T11:19:45.411334+0000","last_fullsized":"2023-11-28T11:19:45.411334+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:16:20.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422230+0000","last_change":"2023-11-28T11:19:44.415253+0000","last_active":"2023-11-28T11:19:44.422230+0000","last_peered":"2023-11-28T11:19:44.422230+0000","last_clean":"2023-11-28T11:19:44.422230+0000","last_became_active":"2023-11-28T11:19:44.414979+0000","last_became_peered":"2023-11-28T11:19:44.414979+0000","last_unstale":"2023-11-28T11:19:44.422230+0000","last_undegraded":"2023-11-28T11:19:44.422230+0000","last_fullsized":"2023-11-28T11:19:44.422230+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:31.471220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629252+0000","last_change":"2023-11-28T11:19:44.421005+0000","last_active":"2023-11-28T11:19:45.629252+0000","last_peered":"2023-11-28T11:19:45.629252+0000","last_clean":"2023-11-28T11:19:45.629252+0000","last_became_active":"2023-11-28T11:19:44.420364+0000","last_became_peered":"2023-11-28T11:19:44.420364+0000","last_unstale":"2023-11-28T11:19:45.629252+0000","last_undegraded":"2023-11-28T11:19:45.629252+0000","last_fullsized":"2023-11-28T11:19:45.629252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:39.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412333+0000","last_change":"2023-11-28T11:19:44.433747+0000","last_active":"2023-11-28T11:19:45.412333+0000","last_peered":"2023-11-28T11:19:45.412333+0000","last_clean":"2023-11-28T11:19:45.412333+0000","last_became_active":"2023-11-28T11:19:44.433472+0000","last_became_peered":"2023-11-28T11:19:44.433472+0000","last_unstale":"2023-11-28T11:19:45.412333+0000","last_undegraded":"2023-11-28T11:19:45.412333+0000","last_fullsized":"2023-11-28T11:19:45.412333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:15.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.417992+0000","last_change":"2023-11-28T11:19:44.411732+0000","last_active":"2023-11-28T11:19:44.417992+0000","last_peered":"2023-11-28T11:19:44.417992+0000","last_clean":"2023-11-28T11:19:44.417992+0000","last_became_active":"2023-11-28T11:19:44.411449+0000","last_became_peered":"2023-11-28T11:19:44.411449+0000","last_unstale":"2023-11-28T11:19:44.417992+0000","last_undegraded":"2023-11-28T11:19:44.417992+0000","last_fullsized":"2023-11-28T11:19:44.417992+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:24.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629795+0000","last_change":"2023-11-28T11:19:44.423670+0000","last_active":"2023-11-28T11:19:45.629795+0000","last_peered":"2023-11-28T11:19:45.629795+0000","last_clean":"2023-11-28T11:19:45.629795+0000","last_became_active":"2023-11-28T11:19:44.423059+0000","last_became_peered":"2023-11-28T11:19:44.423059+0000","last_unstale":"2023-11-28T11:19:45.629795+0000","last_undegraded":"2023-11-28T11:19:45.629795+0000","last_fullsized":"2023-11-28T11:19:45.629795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:51.124532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413124+0000","last_change":"2023-11-28T11:19:44.433802+0000","last_active":"2023-11-28T11:19:45.413124+0000","last_peered":"2023-11-28T11:19:45.413124+0000","last_clean":"2023-11-28T11:19:45.413124+0000","last_became_active":"2023-11-28T11:19:44.433105+0000","last_became_peered":"2023-11-28T11:19:44.433105+0000","last_unstale":"2023-11-28T11:19:45.413124+0000","last_undegraded":"2023-11-28T11:19:45.413124+0000","last_fullsized":"2023-11-28T11:19:45.413124+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:52.159199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629687+0000","last_change":"2023-11-28T11:19:44.425389+0000","last_active":"2023-11-28T11:19:45.629687+0000","last_peered":"2023-11-28T11:19:45.629687+0000","last_clean":"2023-11-28T11:19:45.629687+0000","last_became_active":"2023-11-28T11:19:44.424873+0000","last_became_peered":"2023-11-28T11:19:44.424873+0000","last_unstale":"2023-11-28T11:19:45.629687+0000","last_undegraded":"2023-11-28T11:19:45.629687+0000","last_fullsized":"2023-11-28T11:19:45.629687+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.103036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433585+0000","last_change":"2023-11-28T11:19:44.429138+0000","last_active":"2023-11-28T11:19:44.433585+0000","last_peered":"2023-11-28T11:19:44.433585+0000","last_clean":"2023-11-28T11:19:44.433585+0000","last_became_active":"2023-11-28T11:19:44.424325+0000","last_became_peered":"2023-11-28T11:19:44.424325+0000","last_unstale":"2023-11-28T11:19:44.433585+0000","last_undegraded":"2023-11-28T11:19:44.433585+0000","last_fullsized":"2023-11-28T11:19:44.433585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:10.100216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429042+0000","last_change":"2023-11-28T11:19:44.421827+0000","last_active":"2023-11-28T11:19:44.429042+0000","last_peered":"2023-11-28T11:19:44.429042+0000","last_clean":"2023-11-28T11:19:44.429042+0000","last_became_active":"2023-11-28T11:19:44.421595+0000","last_became_peered":"2023-11-28T11:19:44.421595+0000","last_unstale":"2023-11-28T11:19:44.429042+0000","last_undegraded":"2023-11-28T11:19:44.429042+0000","last_fullsized":"2023-11-28T11:19:44.429042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:27.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410982+0000","last_change":"2023-11-28T11:19:44.438916+0000","last_active":"2023-11-28T11:19:45.410982+0000","last_peered":"2023-11-28T11:19:45.410982+0000","last_clean":"2023-11-28T11:19:45.410982+0000","last_became_active":"2023-11-28T11:19:44.436975+0000","last_became_peered":"2023-11-28T11:19:44.436975+0000","last_unstale":"2023-11-28T11:19:45.410982+0000","last_undegraded":"2023-11-28T11:19:45.410982+0000","last_fullsized":"2023-11-28T11:19:45.410982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:14.235872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.427402+0000","last_change":"2023-11-28T11:19:44.425640+0000","last_active":"2023-11-28T11:19:44.427402+0000","last_peered":"2023-11-28T11:19:44.427402+0000","last_clean":"2023-11-28T11:19:44.427402+0000","last_became_active":"2023-11-28T11:19:44.425382+0000","last_became_peered":"2023-11-28T11:19:44.425382+0000","last_unstale":"2023-11-28T11:19:44.427402+0000","last_undegraded":"2023-11-28T11:19:44.427402+0000","last_fullsized":"2023-11-28T11:19:44.427402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:38.591175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629033+0000","last_change":"2023-11-28T11:19:44.418272+0000","last_active":"2023-11-28T11:19:45.629033+0000","last_peered":"2023-11-28T11:19:45.629033+0000","last_clean":"2023-11-28T11:19:45.629033+0000","last_became_active":"2023-11-28T11:19:44.417864+0000","last_became_peered":"2023-11-28T11:19:44.417864+0000","last_unstale":"2023-11-28T11:19:45.629033+0000","last_undegraded":"2023-11-28T11:19:45.629033+0000","last_fullsized":"2023-11-28T11:19:45.629033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629769+0000","last_change":"2023-11-28T11:19:44.434517+0000","last_active":"2023-11-28T11:19:45.629769+0000","last_peered":"2023-11-28T11:19:45.629769+0000","last_clean":"2023-11-28T11:19:45.629769+0000","last_became_active":"2023-11-28T11:19:44.434266+0000","last_became_peered":"2023-11-28T11:19:44.434266+0000","last_unstale":"2023-11-28T11:19:45.629769+0000","last_undegraded":"2023-11-28T11:19:45.629769+0000","last_fullsized":"2023-11-28T11:19:45.629769+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:38.251735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411240+0000","last_change":"2023-11-28T11:19:44.430179+0000","last_active":"2023-11-28T11:19:45.411240+0000","last_peered":"2023-11-28T11:19:45.411240+0000","last_clean":"2023-11-28T11:19:45.411240+0000","last_became_active":"2023-11-28T11:19:44.429912+0000","last_became_peered":"2023-11-28T11:19:44.429912+0000","last_unstale":"2023-11-28T11:19:45.411240+0000","last_undegraded":"2023-11-28T11:19:45.411240+0000","last_fullsized":"2023-11-28T11:19:45.411240+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436098+0000","last_change":"2023-11-28T11:19:44.433285+0000","last_active":"2023-11-28T11:19:44.436098+0000","last_peered":"2023-11-28T11:19:44.436098+0000","last_clean":"2023-11-28T11:19:44.436098+0000","last_became_active":"2023-11-28T11:19:44.432835+0000","last_became_peered":"2023-11-28T11:19:44.432835+0000","last_unstale":"2023-11-28T11:19:44.436098+0000","last_undegraded":"2023-11-28T11:19:44.436098+0000","last_fullsized":"2023-11-28T11:19:44.436098+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:44.822864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.424505+0000","last_change":"2023-11-28T11:19:44.420034+0000","last_active":"2023-11-28T11:19:44.424505+0000","last_peered":"2023-11-28T11:19:44.424505+0000","last_clean":"2023-11-28T11:19:44.424505+0000","last_became_active":"2023-11-28T11:19:44.419747+0000","last_became_peered":"2023-11-28T11:19:44.419747+0000","last_unstale":"2023-11-28T11:19:44.424505+0000","last_undegraded":"2023-11-28T11:19:44.424505+0000","last_fullsized":"2023-11-28T11:19:44.424505+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:04.071637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629305+0000","last_change":"2023-11-28T11:19:44.419852+0000","last_active":"2023-11-28T11:19:45.629305+0000","last_peered":"2023-11-28T11:19:45.629305+0000","last_clean":"2023-11-28T11:19:45.629305+0000","last_became_active":"2023-11-28T11:19:44.419580+0000","last_became_peered":"2023-11-28T11:19:44.419580+0000","last_unstale":"2023-11-28T11:19:45.629305+0000","last_undegraded":"2023-11-28T11:19:45.629305+0000","last_fullsized":"2023-11-28T11:19:45.629305+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:05.409658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412486+0000","last_change":"2023-11-28T11:19:44.431540+0000","last_active":"2023-11-28T11:19:45.412486+0000","last_peered":"2023-11-28T11:19:45.412486+0000","last_clean":"2023-11-28T11:19:45.412486+0000","last_became_active":"2023-11-28T11:19:44.429838+0000","last_became_peered":"2023-11-28T11:19:44.429838+0000","last_unstale":"2023-11-28T11:19:45.412486+0000","last_undegraded":"2023-11-28T11:19:45.412486+0000","last_fullsized":"2023-11-28T11:19:45.412486+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:15.792643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629203+0000","last_change":"2023-11-28T11:19:44.420982+0000","last_active":"2023-11-28T11:19:45.629203+0000","last_peered":"2023-11-28T11:19:45.629203+0000","last_clean":"2023-11-28T11:19:45.629203+0000","last_became_active":"2023-11-28T11:19:44.420691+0000","last_became_peered":"2023-11-28T11:19:44.420691+0000","last_unstale":"2023-11-28T11:19:45.629203+0000","last_undegraded":"2023-11-28T11:19:45.629203+0000","last_fullsized":"2023-11-28T11:19:45.629203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:30.934789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433617+0000","last_change":"2023-11-28T11:19:44.431282+0000","last_active":"2023-11-28T11:19:44.433617+0000","last_peered":"2023-11-28T11:19:44.433617+0000","last_clean":"2023-11-28T11:19:44.433617+0000","last_became_active":"2023-11-28T11:19:44.430979+0000","last_became_peered":"2023-11-28T11:19:44.430979+0000","last_unstale":"2023-11-28T11:19:44.433617+0000","last_undegraded":"2023-11-28T11:19:44.433617+0000","last_fullsized":"2023-11-28T11:19:44.433617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:29.320378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412012+0000","last_change":"2023-11-28T11:19:44.420787+0000","last_active":"2023-11-28T11:19:45.412012+0000","last_peered":"2023-11-28T11:19:45.412012+0000","last_clean":"2023-11-28T11:19:45.412012+0000","last_became_active":"2023-11-28T11:19:44.420491+0000","last_became_peered":"2023-11-28T11:19:44.420491+0000","last_unstale":"2023-11-28T11:19:45.412012+0000","last_undegraded":"2023-11-28T11:19:45.412012+0000","last_fullsized":"2023-11-28T11:19:45.412012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:26.387178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436077+0000","last_change":"2023-11-28T11:19:43.904752+0000","last_active":"2023-11-28T11:19:44.436077+0000","last_peered":"2023-11-28T11:19:44.436077+0000","last_clean":"2023-11-28T11:19:44.436077+0000","last_became_active":"2023-11-28T11:19:43.904340+0000","last_became_peered":"2023-11-28T11:19:43.904340+0000","last_unstale":"2023-11-28T11:19:44.436077+0000","last_undegraded":"2023-11-28T11:19:44.436077+0000","last_fullsized":"2023-11-28T11:19:44.436077+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:07.203302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433112+0000","last_change":"2023-11-28T11:19:44.431312+0000","last_active":"2023-11-28T11:19:44.433112+0000","last_peered":"2023-11-28T11:19:44.433112+0000","last_clean":"2023-11-28T11:19:44.433112+0000","last_became_active":"2023-11-28T11:19:44.428700+0000","last_became_peered":"2023-11-28T11:19:44.428700+0000","last_unstale":"2023-11-28T11:19:44.433112+0000","last_undegraded":"2023-11-28T11:19:44.433112+0000","last_fullsized":"2023-11-28T11:19:44.433112+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:57.924786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.442196+0000","last_change":"2023-11-28T11:19:43.907309+0000","last_active":"2023-11-28T11:19:44.442196+0000","last_peered":"2023-11-28T11:19:44.442196+0000","last_clean":"2023-11-28T11:19:44.442196+0000","last_became_active":"2023-11-28T11:19:43.907091+0000","last_became_peered":"2023-11-28T11:19:43.907091+0000","last_unstale":"2023-11-28T11:19:44.442196+0000","last_undegraded":"2023-11-28T11:19:44.442196+0000","last_fullsized":"2023-11-28T11:19:44.442196+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:55:49.432119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429901+0000","last_change":"2023-11-28T11:19:44.422736+0000","last_active":"2023-11-28T11:19:44.429901+0000","last_peered":"2023-11-28T11:19:44.429901+0000","last_clean":"2023-11-28T11:19:44.429901+0000","last_became_active":"2023-11-28T11:19:44.422423+0000","last_became_peered":"2023-11-28T11:19:44.422423+0000","last_unstale":"2023-11-28T11:19:44.429901+0000","last_undegraded":"2023-11-28T11:19:44.429901+0000","last_fullsized":"2023-11-28T11:19:44.429901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:21.073037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.443441+0000","last_change":"2023-11-28T11:19:43.915537+0000","last_active":"2023-11-28T11:19:44.443441+0000","last_peered":"2023-11-28T11:19:44.443441+0000","last_clean":"2023-11-28T11:19:44.443441+0000","last_became_active":"2023-11-28T11:19:43.914431+0000","last_became_peered":"2023-11-28T11:19:43.914431+0000","last_unstale":"2023-11-28T11:19:44.443441+0000","last_undegraded":"2023-11-28T11:19:44.443441+0000","last_fullsized":"2023-11-28T11:19:44.443441+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:59.706612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410945+0000","last_change":"2023-11-28T11:19:44.432519+0000","last_active":"2023-11-28T11:19:45.410945+0000","last_peered":"2023-11-28T11:19:45.410945+0000","last_clean":"2023-11-28T11:19:45.410945+0000","last_became_active":"2023-11-28T11:19:44.431956+0000","last_became_peered":"2023-11-28T11:19:44.431956+0000","last_unstale":"2023-11-28T11:19:45.410945+0000","last_undegraded":"2023-11-28T11:19:45.410945+0000","last_fullsized":"2023-11-28T11:19:45.410945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:39.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.440659+0000","last_change":"2023-11-28T11:19:43.906143+0000","last_active":"2023-11-28T11:19:44.440659+0000","last_peered":"2023-11-28T11:19:44.440659+0000","last_clean":"2023-11-28T11:19:44.440659+0000","last_became_active":"2023-11-28T11:19:43.905852+0000","last_became_peered":"2023-11-28T11:19:43.905852+0000","last_unstale":"2023-11-28T11:19:44.440659+0000","last_undegraded":"2023-11-28T11:19:44.440659+0000","last_fullsized":"2023-11-28T11:19:44.440659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:40.554063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413088+0000","last_change":"2023-11-28T11:19:44.417691+0000","last_active":"2023-11-28T11:19:45.413088+0000","last_peered":"2023-11-28T11:19:45.413088+0000","last_clean":"2023-11-28T11:19:45.413088+0000","last_became_active":"2023-11-28T11:19:44.417429+0000","last_became_peered":"2023-11-28T11:19:44.417429+0000","last_unstale":"2023-11-28T11:19:45.413088+0000","last_undegraded":"2023-11-28T11:19:45.413088+0000","last_fullsized":"2023-11-28T11:19:45.413088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629308+0000","last_change":"2023-11-28T11:19:43.910881+0000","last_active":"2023-11-28T11:19:45.629308+0000","last_peered":"2023-11-28T11:19:45.629308+0000","last_clean":"2023-11-28T11:19:45.629308+0000","last_became_active":"2023-11-28T11:19:43.910504+0000","last_became_peered":"2023-11-28T11:19:43.910504+0000","last_unstale":"2023-11-28T11:19:45.629308+0000","last_undegraded":"2023-11-28T11:19:45.629308+0000","last_fullsized":"2023-11-28T11:19:45.629308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:06.139015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412649+0000","last_change":"2023-11-28T11:19:44.437211+0000","last_active":"2023-11-28T11:19:45.412649+0000","last_peered":"2023-11-28T11:19:45.412649+0000","last_clean":"2023-11-28T11:19:45.412649+0000","last_became_active":"2023-11-28T11:19:44.436967+0000","last_became_peered":"2023-11-28T11:19:44.436967+0000","last_unstale":"2023-11-28T11:19:45.412649+0000","last_undegraded":"2023-11-28T11:19:45.412649+0000","last_fullsized":"2023-11-28T11:19:45.412649+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412280+0000","last_change":"2023-11-28T11:19:43.902672+0000","last_active":"2023-11-28T11:19:45.412280+0000","last_peered":"2023-11-28T11:19:45.412280+0000","last_clean":"2023-11-28T11:19:45.412280+0000","last_became_active":"2023-11-28T11:19:43.902370+0000","last_became_peered":"2023-11-28T11:19:43.902370+0000","last_unstale":"2023-11-28T11:19:45.412280+0000","last_undegraded":"2023-11-28T11:19:45.412280+0000","last_fullsized":"2023-11-28T11:19:45.412280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411355+0000","last_change":"2023-11-28T11:19:44.431721+0000","last_active":"2023-11-28T11:19:45.411355+0000","last_peered":"2023-11-28T11:19:45.411355+0000","last_clean":"2023-11-28T11:19:45.411355+0000","last_became_active":"2023-11-28T11:19:44.431423+0000","last_became_peered":"2023-11-28T11:19:44.431423+0000","last_unstale":"2023-11-28T11:19:45.411355+0000","last_undegraded":"2023-11-28T11:19:45.411355+0000","last_fullsized":"2023-11-28T11:19:45.411355+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.413688+0000","last_change":"2023-11-28T11:19:43.909936+0000","last_active":"2023-11-28T11:19:44.413688+0000","last_peered":"2023-11-28T11:19:44.413688+0000","last_clean":"2023-11-28T11:19:44.413688+0000","last_became_active":"2023-11-28T11:19:43.909555+0000","last_became_peered":"2023-11-28T11:19:43.909555+0000","last_unstale":"2023-11-28T11:19:44.413688+0000","last_undegraded":"2023-11-28T11:19:44.413688+0000","last_fullsized":"2023-11-28T11:19:44.413688+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:33.877574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412778+0000","last_change":"2023-11-28T11:19:44.442153+0000","last_active":"2023-11-28T11:19:45.412778+0000","last_peered":"2023-11-28T11:19:45.412778+0000","last_clean":"2023-11-28T11:19:45.412778+0000","last_became_active":"2023-11-28T11:19:44.441884+0000","last_became_peered":"2023-11-28T11:19:44.441884+0000","last_unstale":"2023-11-28T11:19:45.412778+0000","last_undegraded":"2023-11-28T11:19:45.412778+0000","last_fullsized":"2023-11-28T11:19:45.412778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629060+0000","last_change":"2023-11-28T11:19:43.904739+0000","last_active":"2023-11-28T11:19:45.629060+0000","last_peered":"2023-11-28T11:19:45.629060+0000","last_clean":"2023-11-28T11:19:45.629060+0000","last_became_active":"2023-11-28T11:19:43.904476+0000","last_became_peered":"2023-11-28T11:19:43.904476+0000","last_unstale":"2023-11-28T11:19:45.629060+0000","last_undegraded":"2023-11-28T11:19:45.629060+0000","last_fullsized":"2023-11-28T11:19:45.629060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:50.176370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412130+0000","last_change":"2023-11-28T11:19:44.427625+0000","last_active":"2023-11-28T11:19:45.412130+0000","last_peered":"2023-11-28T11:19:45.412130+0000","last_clean":"2023-11-28T11:19:45.412130+0000","last_became_active":"2023-11-28T11:19:44.423771+0000","last_became_peered":"2023-11-28T11:19:44.423771+0000","last_unstale":"2023-11-28T11:19:45.412130+0000","last_undegraded":"2023-11-28T11:19:45.412130+0000","last_fullsized":"2023-11-28T11:19:45.412130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:40:10.501924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412920+0000","last_change":"2023-11-28T11:19:43.911818+0000","last_active":"2023-11-28T11:19:45.412920+0000","last_peered":"2023-11-28T11:19:45.412920+0000","last_clean":"2023-11-28T11:19:45.412920+0000","last_became_active":"2023-11-28T11:19:43.911619+0000","last_became_peered":"2023-11-28T11:19:43.911619+0000","last_unstale":"2023-11-28T11:19:45.412920+0000","last_undegraded":"2023-11-28T11:19:45.412920+0000","last_fullsized":"2023-11-28T11:19:45.412920+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:06.823918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.420172+0000","last_change":"2023-11-28T11:19:44.418304+0000","last_active":"2023-11-28T11:19:44.420172+0000","last_peered":"2023-11-28T11:19:44.420172+0000","last_clean":"2023-11-28T11:19:44.420172+0000","last_became_active":"2023-11-28T11:19:44.414877+0000","last_became_peered":"2023-11-28T11:19:44.414877+0000","last_unstale":"2023-11-28T11:19:44.420172+0000","last_undegraded":"2023-11-28T11:19:44.420172+0000","last_fullsized":"2023-11-28T11:19:44.420172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:33.837740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413074+0000","last_change":"2023-11-28T11:19:43.902678+0000","last_active":"2023-11-28T11:19:45.413074+0000","last_peered":"2023-11-28T11:19:45.413074+0000","last_clean":"2023-11-28T11:19:45.413074+0000","last_became_active":"2023-11-28T11:19:43.902317+0000","last_became_peered":"2023-11-28T11:19:43.902317+0000","last_unstale":"2023-11-28T11:19:45.413074+0000","last_undegraded":"2023-11-28T11:19:45.413074+0000","last_fullsized":"2023-11-28T11:19:45.413074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:24.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629333+0000","last_change":"2023-11-28T11:19:44.420213+0000","last_active":"2023-11-28T11:19:45.629333+0000","last_peered":"2023-11-28T11:19:45.629333+0000","last_clean":"2023-11-28T11:19:45.629333+0000","last_became_active":"2023-11-28T11:19:44.419930+0000","last_became_peered":"2023-11-28T11:19:44.419930+0000","last_unstale":"2023-11-28T11:19:45.629333+0000","last_undegraded":"2023-11-28T11:19:45.629333+0000","last_fullsized":"2023-11-28T11:19:45.629333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:09:01.560420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412679+0000","last_change":"2023-11-28T11:19:43.916715+0000","last_active":"2023-11-28T11:19:45.412679+0000","last_peered":"2023-11-28T11:19:45.412679+0000","last_clean":"2023-11-28T11:19:45.412679+0000","last_became_active":"2023-11-28T11:19:43.916453+0000","last_became_peered":"2023-11-28T11:19:43.916453+0000","last_unstale":"2023-11-28T11:19:45.412679+0000","last_undegraded":"2023-11-28T11:19:45.412679+0000","last_fullsized":"2023-11-28T11:19:45.412679+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:20.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412625+0000","last_change":"2023-11-28T11:19:44.420369+0000","last_active":"2023-11-28T11:19:45.412625+0000","last_peered":"2023-11-28T11:19:45.412625+0000","last_clean":"2023-11-28T11:19:45.412625+0000","last_became_active":"2023-11-28T11:19:44.420113+0000","last_became_peered":"2023-11-28T11:19:44.420113+0000","last_unstale":"2023-11-28T11:19:45.412625+0000","last_undegraded":"2023-11-28T11:19:45.412625+0000","last_fullsized":"2023-11-28T11:19:45.412625+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:52.159199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411336+0000","last_change":"2023-11-28T11:19:43.903511+0000","last_active":"2023-11-28T11:19:45.411336+0000","last_peered":"2023-11-28T11:19:45.411336+0000","last_clean":"2023-11-28T11:19:45.411336+0000","last_became_active":"2023-11-28T11:19:43.902881+0000","last_became_peered":"2023-11-28T11:19:43.902881+0000","last_unstale":"2023-11-28T11:19:45.411336+0000","last_undegraded":"2023-11-28T11:19:45.411336+0000","last_fullsized":"2023-11-28T11:19:45.411336+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:31.447467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.424593+0000","last_change":"2023-11-28T11:19:44.420466+0000","last_active":"2023-11-28T11:19:44.424593+0000","last_peered":"2023-11-28T11:19:44.424593+0000","last_clean":"2023-11-28T11:19:44.424593+0000","last_became_active":"2023-11-28T11:19:44.420122+0000","last_became_peered":"2023-11-28T11:19:44.420122+0000","last_unstale":"2023-11-28T11:19:44.424593+0000","last_undegraded":"2023-11-28T11:19:44.424593+0000","last_fullsized":"2023-11-28T11:19:44.424593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:01.014772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629410+0000","last_change":"2023-11-28T11:19:43.908555+0000","last_active":"2023-11-28T11:19:45.629410+0000","last_peered":"2023-11-28T11:19:45.629410+0000","last_clean":"2023-11-28T11:19:45.629410+0000","last_became_active":"2023-11-28T11:19:43.907872+0000","last_became_peered":"2023-11-28T11:19:43.907872+0000","last_unstale":"2023-11-28T11:19:45.629410+0000","last_undegraded":"2023-11-28T11:19:45.629410+0000","last_fullsized":"2023-11-28T11:19:45.629410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629735+0000","last_change":"2023-11-28T11:19:44.431390+0000","last_active":"2023-11-28T11:19:45.629735+0000","last_peered":"2023-11-28T11:19:45.629735+0000","last_clean":"2023-11-28T11:19:45.629735+0000","last_became_active":"2023-11-28T11:19:44.431077+0000","last_became_peered":"2023-11-28T11:19:44.431077+0000","last_unstale":"2023-11-28T11:19:45.629735+0000","last_undegraded":"2023-11-28T11:19:45.629735+0000","last_fullsized":"2023-11-28T11:19:45.629735+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.107723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.415912+0000","last_change":"2023-11-28T11:19:43.914509+0000","last_active":"2023-11-28T11:19:44.415912+0000","last_peered":"2023-11-28T11:19:44.415912+0000","last_clean":"2023-11-28T11:19:44.415912+0000","last_became_active":"2023-11-28T11:19:43.914224+0000","last_became_peered":"2023-11-28T11:19:43.914224+0000","last_unstale":"2023-11-28T11:19:44.415912+0000","last_undegraded":"2023-11-28T11:19:44.415912+0000","last_fullsized":"2023-11-28T11:19:44.415912+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:03.564484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411307+0000","last_change":"2023-11-28T11:19:44.432262+0000","last_active":"2023-11-28T11:19:45.411307+0000","last_peered":"2023-11-28T11:19:45.411307+0000","last_clean":"2023-11-28T11:19:45.411307+0000","last_became_active":"2023-11-28T11:19:44.431980+0000","last_became_peered":"2023-11-28T11:19:44.431980+0000","last_unstale":"2023-11-28T11:19:45.411307+0000","last_undegraded":"2023-11-28T11:19:45.411307+0000","last_fullsized":"2023-11-28T11:19:45.411307+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:13.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412415+0000","last_change":"2023-11-28T11:19:43.911385+0000","last_active":"2023-11-28T11:19:45.412415+0000","last_peered":"2023-11-28T11:19:45.412415+0000","last_clean":"2023-11-28T11:19:45.412415+0000","last_became_active":"2023-11-28T11:19:43.911133+0000","last_became_peered":"2023-11-28T11:19:43.911133+0000","last_unstale":"2023-11-28T11:19:45.412415+0000","last_undegraded":"2023-11-28T11:19:45.412415+0000","last_fullsized":"2023-11-28T11:19:45.412415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:03.066249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629652+0000","last_change":"2023-11-28T11:19:44.423876+0000","last_active":"2023-11-28T11:19:45.629652+0000","last_peered":"2023-11-28T11:19:45.629652+0000","last_clean":"2023-11-28T11:19:45.629652+0000","last_became_active":"2023-11-28T11:19:44.423428+0000","last_became_peered":"2023-11-28T11:19:44.423428+0000","last_unstale":"2023-11-28T11:19:45.629652+0000","last_undegraded":"2023-11-28T11:19:45.629652+0000","last_fullsized":"2023-11-28T11:19:45.629652+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:14.235872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411011+0000","last_change":"2023-11-28T11:19:43.901095+0000","last_active":"2023-11-28T11:19:45.411011+0000","last_peered":"2023-11-28T11:19:45.411011+0000","last_clean":"2023-11-28T11:19:45.411011+0000","last_became_active":"2023-11-28T11:19:43.900868+0000","last_became_peered":"2023-11-28T11:19:43.900868+0000","last_unstale":"2023-11-28T11:19:45.411011+0000","last_undegraded":"2023-11-28T11:19:45.411011+0000","last_fullsized":"2023-11-28T11:19:45.411011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:06.253620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410912+0000","last_change":"2023-11-28T11:19:44.435236+0000","last_active":"2023-11-28T11:19:45.410912+0000","last_peered":"2023-11-28T11:19:45.410912+0000","last_clean":"2023-11-28T11:19:45.410912+0000","last_became_active":"2023-11-28T11:19:44.434959+0000","last_became_peered":"2023-11-28T11:19:44.434959+0000","last_unstale":"2023-11-28T11:19:45.410912+0000","last_undegraded":"2023-11-28T11:19:45.410912+0000","last_fullsized":"2023-11-28T11:19:45.410912+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:25.634003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629658+0000","last_change":"2023-11-28T11:19:43.916369+0000","last_active":"2023-11-28T11:19:45.629658+0000","last_peered":"2023-11-28T11:19:45.629658+0000","last_clean":"2023-11-28T11:19:45.629658+0000","last_became_active":"2023-11-28T11:19:43.916103+0000","last_became_peered":"2023-11-28T11:19:43.916103+0000","last_unstale":"2023-11-28T11:19:45.629658+0000","last_undegraded":"2023-11-28T11:19:45.629658+0000","last_fullsized":"2023-11-28T11:19:45.629658+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:06.253620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429932+0000","last_change":"2023-11-28T11:19:44.427952+0000","last_active":"2023-11-28T11:19:44.429932+0000","last_peered":"2023-11-28T11:19:44.429932+0000","last_clean":"2023-11-28T11:19:44.429932+0000","last_became_active":"2023-11-28T11:19:44.427784+0000","last_became_peered":"2023-11-28T11:19:44.427784+0000","last_unstale":"2023-11-28T11:19:44.429932+0000","last_undegraded":"2023-11-28T11:19:44.429932+0000","last_fullsized":"2023-11-28T11:19:44.429932+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:41.869600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.435875+0000","last_change":"2023-11-28T11:19:43.908328+0000","last_active":"2023-11-28T11:19:44.435875+0000","last_peered":"2023-11-28T11:19:44.435875+0000","last_clean":"2023-11-28T11:19:44.435875+0000","last_became_active":"2023-11-28T11:19:43.908027+0000","last_became_peered":"2023-11-28T11:19:43.908027+0000","last_unstale":"2023-11-28T11:19:44.435875+0000","last_undegraded":"2023-11-28T11:19:44.435875+0000","last_fullsized":"2023-11-28T11:19:44.435875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:13.290188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412171+0000","last_change":"2023-11-28T11:19:44.421452+0000","last_active":"2023-11-28T11:19:45.412171+0000","last_peered":"2023-11-28T11:19:45.412171+0000","last_clean":"2023-11-28T11:19:45.412171+0000","last_became_active":"2023-11-28T11:19:44.421198+0000","last_became_peered":"2023-11-28T11:19:44.421198+0000","last_unstale":"2023-11-28T11:19:45.412171+0000","last_undegraded":"2023-11-28T11:19:45.412171+0000","last_fullsized":"2023-11-28T11:19:45.412171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436343+0000","last_change":"2023-11-28T11:19:43.915495+0000","last_active":"2023-11-28T11:19:44.436343+0000","last_peered":"2023-11-28T11:19:44.436343+0000","last_clean":"2023-11-28T11:19:44.436343+0000","last_became_active":"2023-11-28T11:19:43.915226+0000","last_became_peered":"2023-11-28T11:19:43.915226+0000","last_unstale":"2023-11-28T11:19:44.436343+0000","last_undegraded":"2023-11-28T11:19:44.436343+0000","last_fullsized":"2023-11-28T11:19:44.436343+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.585787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422212+0000","last_change":"2023-11-28T11:19:44.412664+0000","last_active":"2023-11-28T11:19:44.422212+0000","last_peered":"2023-11-28T11:19:44.422212+0000","last_clean":"2023-11-28T11:19:44.422212+0000","last_became_active":"2023-11-28T11:19:44.412425+0000","last_became_peered":"2023-11-28T11:19:44.412425+0000","last_unstale":"2023-11-28T11:19:44.422212+0000","last_undegraded":"2023-11-28T11:19:44.422212+0000","last_fullsized":"2023-11-28T11:19:44.422212+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:57.310680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.415421+0000","last_change":"2023-11-28T11:19:43.907529+0000","last_active":"2023-11-28T11:19:44.415421+0000","last_peered":"2023-11-28T11:19:44.415421+0000","last_clean":"2023-11-28T11:19:44.415421+0000","last_became_active":"2023-11-28T11:19:43.907238+0000","last_became_peered":"2023-11-28T11:19:43.907238+0000","last_unstale":"2023-11-28T11:19:44.415421+0000","last_undegraded":"2023-11-28T11:19:44.415421+0000","last_fullsized":"2023-11-28T11:19:44.415421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:13.290188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.418211+0000","last_change":"2023-11-28T11:19:44.414611+0000","last_active":"2023-11-28T11:19:44.418211+0000","last_peered":"2023-11-28T11:19:44.418211+0000","last_clean":"2023-11-28T11:19:44.418211+0000","last_became_active":"2023-11-28T11:19:44.414348+0000","last_became_peered":"2023-11-28T11:19:44.414348+0000","last_unstale":"2023-11-28T11:19:44.418211+0000","last_undegraded":"2023-11-28T11:19:44.418211+0000","last_fullsized":"2023-11-28T11:19:44.418211+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.308677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.440307+0000","last_change":"2023-11-28T11:19:43.915646+0000","last_active":"2023-11-28T11:19:44.440307+0000","last_peered":"2023-11-28T11:19:44.440307+0000","last_clean":"2023-11-28T11:19:44.440307+0000","last_became_active":"2023-11-28T11:19:43.915261+0000","last_became_peered":"2023-11-28T11:19:43.915261+0000","last_unstale":"2023-11-28T11:19:44.440307+0000","last_undegraded":"2023-11-28T11:19:44.440307+0000","last_fullsized":"2023-11-28T11:19:44.440307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:55.937899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.420802+0000","last_change":"2023-11-28T11:19:44.418876+0000","last_active":"2023-11-28T11:19:44.420802+0000","last_peered":"2023-11-28T11:19:44.420802+0000","last_clean":"2023-11-28T11:19:44.420802+0000","last_became_active":"2023-11-28T11:19:44.418558+0000","last_became_peered":"2023-11-28T11:19:44.418558+0000","last_unstale":"2023-11-28T11:19:44.420802+0000","last_undegraded":"2023-11-28T11:19:44.420802+0000","last_fullsized":"2023-11-28T11:19:44.420802+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:53.926768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410994+0000","last_change":"2023-11-28T11:19:43.901092+0000","last_active":"2023-11-28T11:19:45.410994+0000","last_peered":"2023-11-28T11:19:45.410994+0000","last_clean":"2023-11-28T11:19:45.410994+0000","last_became_active":"2023-11-28T11:19:43.900822+0000","last_became_peered":"2023-11-28T11:19:43.900822+0000","last_unstale":"2023-11-28T11:19:45.410994+0000","last_undegraded":"2023-11-28T11:19:45.410994+0000","last_fullsized":"2023-11-28T11:19:45.410994+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629555+0000","last_change":"2023-11-28T11:19:44.426361+0000","last_active":"2023-11-28T11:19:45.629555+0000","last_peered":"2023-11-28T11:19:45.629555+0000","last_clean":"2023-11-28T11:19:45.629555+0000","last_became_active":"2023-11-28T11:19:44.425831+0000","last_became_peered":"2023-11-28T11:19:44.425831+0000","last_unstale":"2023-11-28T11:19:45.629555+0000","last_undegraded":"2023-11-28T11:19:45.629555+0000","last_fullsized":"2023-11-28T11:19:45.629555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:16.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629138+0000","last_change":"2023-11-28T11:19:43.915247+0000","last_active":"2023-11-28T11:19:45.629138+0000","last_peered":"2023-11-28T11:19:45.629138+0000","last_clean":"2023-11-28T11:19:45.629138+0000","last_became_active":"2023-11-28T11:19:43.914898+0000","last_became_peered":"2023-11-28T11:19:43.914898+0000","last_unstale":"2023-11-28T11:19:45.629138+0000","last_undegraded":"2023-11-28T11:19:45.629138+0000","last_fullsized":"2023-11-28T11:19:45.629138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:24.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.430466+0000","last_change":"2023-11-28T11:19:44.429505+0000","last_active":"2023-11-28T11:19:44.430466+0000","last_peered":"2023-11-28T11:19:44.430466+0000","last_clean":"2023-11-28T11:19:44.430466+0000","last_became_active":"2023-11-28T11:19:44.429216+0000","last_became_peered":"2023-11-28T11:19:44.429216+0000","last_unstale":"2023-11-28T11:19:44.430466+0000","last_undegraded":"2023-11-28T11:19:44.430466+0000","last_fullsized":"2023-11-28T11:19:44.430466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:51.181759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.411363+0000","last_change":"2023-11-28T11:19:43.908495+0000","last_active":"2023-11-28T11:19:44.411363+0000","last_peered":"2023-11-28T11:19:44.411363+0000","last_clean":"2023-11-28T11:19:44.411363+0000","last_became_active":"2023-11-28T11:19:43.908190+0000","last_became_peered":"2023-11-28T11:19:43.908190+0000","last_unstale":"2023-11-28T11:19:44.411363+0000","last_undegraded":"2023-11-28T11:19:44.411363+0000","last_fullsized":"2023-11-28T11:19:44.411363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:07.203302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629568+0000","last_change":"2023-11-28T11:19:44.426534+0000","last_active":"2023-11-28T11:19:45.629568+0000","last_peered":"2023-11-28T11:19:45.629568+0000","last_clean":"2023-11-28T11:19:45.629568+0000","last_became_active":"2023-11-28T11:19:44.426116+0000","last_became_peered":"2023-11-28T11:19:44.426116+0000","last_unstale":"2023-11-28T11:19:45.629568+0000","last_undegraded":"2023-11-28T11:19:45.629568+0000","last_fullsized":"2023-11-28T11:19:45.629568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:25:21.326185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412251+0000","last_change":"2023-11-28T11:19:43.908193+0000","last_active":"2023-11-28T11:19:45.412251+0000","last_peered":"2023-11-28T11:19:45.412251+0000","last_clean":"2023-11-28T11:19:45.412251+0000","last_became_active":"2023-11-28T11:19:43.905448+0000","last_became_peered":"2023-11-28T11:19:43.905448+0000","last_unstale":"2023-11-28T11:19:45.412251+0000","last_undegraded":"2023-11-28T11:19:45.412251+0000","last_fullsized":"2023-11-28T11:19:45.412251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.915245+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:19:43.915245+0000","last_peered":"2023-11-28T11:19:43.915245+0000","last_clean":"2023-11-28T11:19:43.915245+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:19:43.915245+0000","last_undegraded":"2023-11-28T11:19:43.915245+0000","last_fullsized":"2023-11-28T11:19:43.915245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433596+0000","last_change":"2023-11-28T11:19:44.431134+0000","last_active":"2023-11-28T11:19:44.433596+0000","last_peered":"2023-11-28T11:19:44.433596+0000","last_clean":"2023-11-28T11:19:44.433596+0000","last_became_active":"2023-11-28T11:19:44.430858+0000","last_became_peered":"2023-11-28T11:19:44.430858+0000","last_unstale":"2023-11-28T11:19:44.433596+0000","last_undegraded":"2023-11-28T11:19:44.433596+0000","last_fullsized":"2023-11-28T11:19:44.433596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:41.514908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.444709+0000","last_change":"2023-11-28T11:19:43.910068+0000","last_active":"2023-11-28T11:19:44.444709+0000","last_peered":"2023-11-28T11:19:44.444709+0000","last_clean":"2023-11-28T11:19:44.444709+0000","last_became_active":"2023-11-28T11:19:43.909181+0000","last_became_peered":"2023-11-28T11:19:43.909181+0000","last_unstale":"2023-11-28T11:19:44.444709+0000","last_undegraded":"2023-11-28T11:19:44.444709+0000","last_fullsized":"2023-11-28T11:19:44.444709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:54.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.900652+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:19:43.900652+0000","last_peered":"2023-11-28T11:19:43.900652+0000","last_clean":"2023-11-28T11:19:43.900652+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:19:43.900652+0000","last_undegraded":"2023-11-28T11:19:43.900652+0000","last_fullsized":"2023-11-28T11:19:43.900652+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411069+0000","last_change":"2023-11-28T11:19:44.432001+0000","last_active":"2023-11-28T11:19:45.411069+0000","last_peered":"2023-11-28T11:19:45.411069+0000","last_clean":"2023-11-28T11:19:45.411069+0000","last_became_active":"2023-11-28T11:19:44.431723+0000","last_became_peered":"2023-11-28T11:19:44.431723+0000","last_unstale":"2023-11-28T11:19:45.411069+0000","last_undegraded":"2023-11-28T11:19:45.411069+0000","last_fullsized":"2023-11-28T11:19:45.411069+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:30.493421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629501+0000","last_change":"2023-11-28T11:19:43.910130+0000","last_active":"2023-11-28T11:19:45.629501+0000","last_peered":"2023-11-28T11:19:45.629501+0000","last_clean":"2023-11-28T11:19:45.629501+0000","last_became_active":"2023-11-28T11:19:43.907373+0000","last_became_peered":"2023-11-28T11:19:43.907373+0000","last_unstale":"2023-11-28T11:19:45.629501+0000","last_undegraded":"2023-11-28T11:19:45.629501+0000","last_fullsized":"2023-11-28T11:19:45.629501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.892135+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:19:43.892135+0000","last_peered":"2023-11-28T11:19:43.892135+0000","last_clean":"2023-11-28T11:19:43.892135+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:19:43.892135+0000","last_undegraded":"2023-11-28T11:19:43.892135+0000","last_fullsized":"2023-11-28T11:19:43.892135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429021+0000","last_change":"2023-11-28T11:19:44.424224+0000","last_active":"2023-11-28T11:19:44.429021+0000","last_peered":"2023-11-28T11:19:44.429021+0000","last_clean":"2023-11-28T11:19:44.429021+0000","last_became_active":"2023-11-28T11:19:44.423861+0000","last_became_peered":"2023-11-28T11:19:44.423861+0000","last_unstale":"2023-11-28T11:19:44.429021+0000","last_undegraded":"2023-11-28T11:19:44.429021+0000","last_fullsized":"2023-11-28T11:19:44.429021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:48.741292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629626+0000","last_change":"2023-11-28T11:19:43.909558+0000","last_active":"2023-11-28T11:19:45.629626+0000","last_peered":"2023-11-28T11:19:45.629626+0000","last_clean":"2023-11-28T11:19:45.629626+0000","last_became_active":"2023-11-28T11:19:43.909307+0000","last_became_peered":"2023-11-28T11:19:43.909307+0000","last_unstale":"2023-11-28T11:19:45.629626+0000","last_undegraded":"2023-11-28T11:19:45.629626+0000","last_fullsized":"2023-11-28T11:19:45.629626+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.900878+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:19:43.900878+0000","last_peered":"2023-11-28T11:19:43.900878+0000","last_clean":"2023-11-28T11:19:43.900878+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:19:43.900878+0000","last_undegraded":"2023-11-28T11:19:43.900878+0000","last_fullsized":"2023-11-28T11:19:43.900878+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412139+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:19:45.412139+0000","last_peered":"2023-11-28T11:19:45.412139+0000","last_clean":"2023-11-28T11:19:45.412139+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:19:45.412139+0000","last_undegraded":"2023-11-28T11:19:45.412139+0000","last_fullsized":"2023-11-28T11:19:45.412139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.421640+0000","last_change":"2023-11-28T11:19:44.418894+0000","last_active":"2023-11-28T11:19:44.421640+0000","last_peered":"2023-11-28T11:19:44.421640+0000","last_clean":"2023-11-28T11:19:44.421640+0000","last_became_active":"2023-11-28T11:19:44.418485+0000","last_became_peered":"2023-11-28T11:19:44.418485+0000","last_unstale":"2023-11-28T11:19:44.421640+0000","last_undegraded":"2023-11-28T11:19:44.421640+0000","last_fullsized":"2023-11-28T11:19:44.421640+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:32.452856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412820+0000","last_change":"2023-11-28T11:19:43.911892+0000","last_active":"2023-11-28T11:19:45.412820+0000","last_peered":"2023-11-28T11:19:45.412820+0000","last_clean":"2023-11-28T11:19:45.412820+0000","last_became_active":"2023-11-28T11:19:43.911116+0000","last_became_peered":"2023-11-28T11:19:43.911116+0000","last_unstale":"2023-11-28T11:19:45.412820+0000","last_undegraded":"2023-11-28T11:19:45.412820+0000","last_fullsized":"2023-11-28T11:19:45.412820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:23.854989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412212+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:19:45.412212+0000","last_peered":"2023-11-28T11:19:45.412212+0000","last_clean":"2023-11-28T11:19:45.412212+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:19:45.412212+0000","last_undegraded":"2023-11-28T11:19:45.412212+0000","last_fullsized":"2023-11-28T11:19:45.412212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411402+0000","last_change":"2023-11-28T11:19:44.429036+0000","last_active":"2023-11-28T11:19:45.411402+0000","last_peered":"2023-11-28T11:19:45.411402+0000","last_clean":"2023-11-28T11:19:45.411402+0000","last_became_active":"2023-11-28T11:19:44.428601+0000","last_became_peered":"2023-11-28T11:19:44.428601+0000","last_unstale":"2023-11-28T11:19:45.411402+0000","last_undegraded":"2023-11-28T11:19:45.411402+0000","last_fullsized":"2023-11-28T11:19:45.411402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:26.544984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.410563+0000","last_change":"2023-11-28T11:19:43.913644+0000","last_active":"2023-11-28T11:19:44.410563+0000","last_peered":"2023-11-28T11:19:44.410563+0000","last_clean":"2023-11-28T11:19:44.410563+0000","last_became_active":"2023-11-28T11:19:43.913370+0000","last_became_peered":"2023-11-28T11:19:43.913370+0000","last_unstale":"2023-11-28T11:19:44.410563+0000","last_undegraded":"2023-11-28T11:19:44.410563+0000","last_fullsized":"2023-11-28T11:19:44.410563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:44.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422285+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:19:44.422285+0000","last_peered":"2023-11-28T11:19:44.422285+0000","last_clean":"2023-11-28T11:19:44.422285+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:19:44.422285+0000","last_undegraded":"2023-11-28T11:19:44.422285+0000","last_fullsized":"2023-11-28T11:19:44.422285+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629408+0000","last_change":"2023-11-28T11:19:44.443917+0000","last_active":"2023-11-28T11:19:45.629408+0000","last_peered":"2023-11-28T11:19:45.629408+0000","last_clean":"2023-11-28T11:19:45.629408+0000","last_became_active":"2023-11-28T11:19:44.440839+0000","last_became_peered":"2023-11-28T11:19:44.440839+0000","last_unstale":"2023-11-28T11:19:45.629408+0000","last_undegraded":"2023-11-28T11:19:45.629408+0000","last_fullsized":"2023-11-28T11:19:45.629408+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:38.183285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411103+0000","last_change":"2023-11-28T11:19:43.902283+0000","last_active":"2023-11-28T11:19:45.411103+0000","last_peered":"2023-11-28T11:19:45.411103+0000","last_clean":"2023-11-28T11:19:45.411103+0000","last_became_active":"2023-11-28T11:19:43.901683+0000","last_became_peered":"2023-11-28T11:19:43.901683+0000","last_unstale":"2023-11-28T11:19:45.411103+0000","last_undegraded":"2023-11-28T11:19:45.411103+0000","last_fullsized":"2023-11-28T11:19:45.411103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:50.176370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.915355+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:19:43.915355+0000","last_peered":"2023-11-28T11:19:43.915355+0000","last_clean":"2023-11-28T11:19:43.915355+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:19:43.915355+0000","last_undegraded":"2023-11-28T11:19:43.915355+0000","last_fullsized":"2023-11-28T11:19:43.915355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.432126+0000","last_change":"2023-11-28T11:19:44.429559+0000","last_active":"2023-11-28T11:19:44.432126+0000","last_peered":"2023-11-28T11:19:44.432126+0000","last_clean":"2023-11-28T11:19:44.432126+0000","last_became_active":"2023-11-28T11:19:44.429301+0000","last_became_peered":"2023-11-28T11:19:44.429301+0000","last_unstale":"2023-11-28T11:19:44.432126+0000","last_undegraded":"2023-11-28T11:19:44.432126+0000","last_fullsized":"2023-11-28T11:19:44.432126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:59.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411078+0000","last_change":"2023-11-28T11:19:43.910348+0000","last_active":"2023-11-28T11:19:45.411078+0000","last_peered":"2023-11-28T11:19:45.411078+0000","last_clean":"2023-11-28T11:19:45.411078+0000","last_became_active":"2023-11-28T11:19:43.910058+0000","last_became_peered":"2023-11-28T11:19:43.910058+0000","last_unstale":"2023-11-28T11:19:45.411078+0000","last_undegraded":"2023-11-28T11:19:45.411078+0000","last_fullsized":"2023-11-28T11:19:45.411078+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.897929+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:19:43.897929+0000","last_peered":"2023-11-28T11:19:43.897929+0000","last_clean":"2023-11-28T11:19:43.897929+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:19:43.897929+0000","last_undegraded":"2023-11-28T11:19:43.897929+0000","last_fullsized":"2023-11-28T11:19:43.897929+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14264,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357576,"statfs":{"total":96636764160,"available":96622157824,"internally_reserved":0,"allocated":1163264,"data_stored":831812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78700000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66100000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88400000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.69299999999999995},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.92800000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96799999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97699999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.747},{"interface":"front","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53700000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12904,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358936,"statfs":{"total":96636764160,"available":96623550464,"internally_reserved":0,"allocated":557056,"data_stored":235303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.53500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":2.23,"15min":2.23},"last":0.84199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79300000000000004,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83999999999999997,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94199999999999995,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.56699999999999995,"5min":0.499,"15min":0.499},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82399999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90400000000000003,"5min":1.627,"15min":1.627},"last":0.88}]},{"osd":4,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96599999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90700000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.55000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12964,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94358876,"statfs":{"total":96636764160,"available":96623489024,"internally_reserved":0,"allocated":552960,"data_stored":233863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56100000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.495},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46400000000000002,"15min":0.46700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95999999999999996,"5min":1.181,"15min":1.181},"last":0.83099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.57599999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86299999999999999,"5min":1.774,"15min":1.774},"last":0.86299999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.753},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97899999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.052,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90700000000000003,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.90200000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14120,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94357720,"statfs":{"total":96636764160,"available":96622305280,"internally_reserved":0,"allocated":1146880,"data_stored":824072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61199999999999999,"5min":1.052,"15min":1.052},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69199999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61199999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.056,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.629},{"interface":"front","average":{"1min":0.754,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.163,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.022,"5min":1.458,"15min":1.458},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.98299999999999998,"5min":1.026,"15min":1.026},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.58899999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.129,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.755},{"interface":"front","average":{"1min":0.745,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1419999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12908,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358932,"statfs":{"total":96636764160,"available":96623546368,"internally_reserved":0,"allocated":561152,"data_stored":236215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.42399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.4580000000000002,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.63100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13244,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358596,"statfs":{"total":96636764160,"available":96623202304,"internally_reserved":0,"allocated":577536,"data_stored":242890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.625,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47099999999999997,"15min":0.47699999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66600000000000004,"5min":3.004,"15min":3.004},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.628,"15min":0.65200000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69299999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":3.274,"15min":3.274},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.52900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.095,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.91800000000000004,"15min":0.94599999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.014,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.81999999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.91900000000000004,"15min":0.94899999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0289999999999999,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0649999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.622,"15min":0.622},"max":{"1min":1.004,"5min":1.099,"15min":1.099},"last":0.88400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13232,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358608,"statfs":{"total":96636764160,"available":96623214592,"internally_reserved":0,"allocated":565248,"data_stored":241886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.56200000000000006}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3720000000000001,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.877,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.002,"5min":1.847,"15min":1.847},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":1.615,"15min":1.615},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.127,"15min":1.127},"last":0.72899999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.55000000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0329999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97899999999999998,"5min":1.296,"15min":1.296},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93799999999999994,"5min":1.212,"15min":1.212},"last":0.503}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13088,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358752,"statfs":{"total":96636764160,"available":96623362048,"internally_reserved":0,"allocated":548864,"data_stored":234667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.68899999999999995,"15min":0.66500000000000004},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72499999999999998,"15min":0.73599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.367,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.442}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.3199999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":2.661,"15min":2.661},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.375,"5min":5.375,"15min":5.375},"last":0.624},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.73599999999999999,"15min":0.72499999999999998},"min":{"1min":0.439,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.76200000000000001,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.333,"15min":1.333},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.1859999999999999,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:19:48.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:19:48.871 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-11-28T11:19:48.870+0000 7f764b1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:19:49.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.063+0000 7f9a4f1ff700 1 Processor -- start 2023-11-28T11:19:49.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.063+0000 7f9a4f1ff700 1 -- start start 2023-11-28T11:19:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a4815c9b0 0x7f9a480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a48161340 0x7f9a480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a480d2320 0x7f9a480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480b5800 con 0x7f9a480d2320 2023-11-28T11:19:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480d1950 con 0x7f9a4815c9b0 2023-11-28T11:19:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480b5520 con 0x7f9a48161340 2023-11-28T11:19:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.064+0000 7f9a4d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a480d2320 0x7f9a480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a480d2320 0x7f9a480cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59532/0 (socket says 172.21.15.53:59532) 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 -- 172.21.15.53:0/3074766572 learned_addr learned my addr 172.21.15.53:0/3074766572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a47fff700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a48161340 0x7f9a480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4cf9b700 1 --2- 172.21.15.53:0/3074766572 >> v2:172.21.15.157:3300/0 conn(0x7f9a4815c9b0 0x7f9a480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 -- 172.21.15.53:0/3074766572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a48161340 msgr2=0x7f9a480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a48161340 0x7f9a480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 -- 172.21.15.53:0/3074766572 >> v2:172.21.15.157:3300/0 conn(0x7f9a4815c9b0 msgr2=0x7f9a480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/3074766572 >> v2:172.21.15.157:3300/0 conn(0x7f9a4815c9b0 0x7f9a480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 -- 172.21.15.53:0/3074766572 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a480d1300 con 0x7f9a480d2320 2023-11-28T11:19:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a480d2320 0x7f9a480cd7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a48067460 tx=0x7f9a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fd03817df1a489 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.065+0000 7f9a477fe700 1 -- 172.21.15.53:0/3074766572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000e070 con 0x7f9a480d2320 2023-11-28T11:19:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a477fe700 1 -- 172.21.15.53:0/3074766572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000b910 con 0x7f9a480d2320 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a477fe700 1 -- 172.21.15.53:0/3074766572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000bcc0 con 0x7f9a480d2320 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/3074766572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a480d2320 msgr2=0x7f9a480cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a480d2320 0x7f9a480cd7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a48067460 tx=0x7f9a4000b330 comp rx=0 tx=0).stop 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/3074766572 shutdown_connections 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a480d2320 0x7f9a480cd7c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/3074766572 >> v2:172.21.15.157:3300/0 conn(0x7f9a4815c9b0 0x7f9a480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/3074766572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a48161340 0x7f9a480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/3074766572 >> 172.21.15.53:0/3074766572 conn(0x7f9a48072600 msgr2=0x7f9a480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/3074766572 shutdown_connections 2023-11-28T11:19:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.066+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/3074766572 wait complete. 2023-11-28T11:19:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.068+0000 7f9a4f1ff700 1 Processor -- start 2023-11-28T11:19:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.068+0000 7f9a4f1ff700 1 -- start start 2023-11-28T11:19:49.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a480d2320 0x7f9a4814da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 0x7f9a48151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a481559b0 con 0x7f9a48161340 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4814dfb0 con 0x7f9a4815c9b0 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48154da0 con 0x7f9a480d2320 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4d79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4d79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59538/0 (socket says 172.21.15.53:59538) 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.069+0000 7f9a4d79c700 1 -- 172.21.15.53:0/1742426035 learned_addr learned my addr 172.21.15.53:0/1742426035 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4cf9b700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a480d2320 0x7f9a4814da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 -- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a480d2320 msgr2=0x7f9a4814da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a480d2320 0x7f9a4814da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 -- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 msgr2=0x7f9a48151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a47fff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 0x7f9a48151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 0x7f9a48151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 -- 172.21.15.53:0/1742426035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40009b30 con 0x7f9a48161340 2023-11-28T11:19:49.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a47fff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 0x7f9a48151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:49.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4d79c700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b680 tx=0x7f9a40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000ce80 con 0x7f9a48161340 2023-11-28T11:19:49.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a40004250 con 0x7f9a48161340 2023-11-28T11:19:49.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a480b5130 con 0x7f9a48161340 2023-11-28T11:19:49.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40004600 con 0x7f9a48161340 2023-11-28T11:19:49.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.070+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a480d15e0 con 0x7f9a48161340 2023-11-28T11:19:49.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.071+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a480b52c0 con 0x7f9a48161340 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.072+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a4002b070 con 0x7f9a48161340 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.073+0000 7f9a457fa700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.073+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a40087040 con 0x7f9a48161340 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.074+0000 7f9a4cf9b700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.075+0000 7f9a4cf9b700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009a10 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.077+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a400b85e0 con 0x7f9a48161340 2023-11-28T11:19:49.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.273+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a480672b0 con 0x7f9a2c0682f0 2023-11-28T11:19:49.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.283+0000 7f9a457fa700 1 -- 172.21.15.53:0/1742426035 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311011 (secure 0 0 0) 0x7f9a480672b0 con 0x7f9a2c0682f0 2023-11-28T11:19:49.287 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 msgr2=0x7f9a2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009a10 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 msgr2=0x7f9a4815ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b680 tx=0x7f9a40004ad0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 shutdown_connections 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a480d2320 0x7f9a4814da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a48161340 0x7f9a4815ec60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 --2- 172.21.15.53:0/1742426035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a4815c9b0 0x7f9a48151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 >> 172.21.15.53:0/1742426035 conn(0x7f9a48072600 msgr2=0x7f9a4806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 shutdown_connections 2023-11-28T11:19:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.287+0000 7f9a4f1ff700 1 -- 172.21.15.53:0/1742426035 wait complete. 2023-11-28T11:19:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:19:49.306 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-11-28T11:19:48.960465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":106724,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":101120,"kb_avail":754867996,"statfs":{"total":773094113280,"available":772984827904,"internally_reserved":0,"allocated":5672960,"data_stored":3080708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":103546880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001594"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.428394+0000","last_change":"2023-11-28T11:19:44.421978+0000","last_active":"2023-11-28T11:19:44.428394+0000","last_peered":"2023-11-28T11:19:44.428394+0000","last_clean":"2023-11-28T11:19:44.428394+0000","last_became_active":"2023-11-28T11:19:44.421712+0000","last_became_peered":"2023-11-28T11:19:44.421712+0000","last_unstale":"2023-11-28T11:19:44.428394+0000","last_undegraded":"2023-11-28T11:19:44.428394+0000","last_fullsized":"2023-11-28T11:19:44.428394+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:17.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412996+0000","last_change":"2023-11-28T11:19:44.433902+0000","last_active":"2023-11-28T11:19:45.412996+0000","last_peered":"2023-11-28T11:19:45.412996+0000","last_clean":"2023-11-28T11:19:45.412996+0000","last_became_active":"2023-11-28T11:19:44.432689+0000","last_became_peered":"2023-11-28T11:19:44.432689+0000","last_unstale":"2023-11-28T11:19:45.412996+0000","last_undegraded":"2023-11-28T11:19:45.412996+0000","last_fullsized":"2023-11-28T11:19:45.412996+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:40:10.501924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629499+0000","last_change":"2023-11-28T11:19:44.425167+0000","last_active":"2023-11-28T11:19:45.629499+0000","last_peered":"2023-11-28T11:19:45.629499+0000","last_clean":"2023-11-28T11:19:45.629499+0000","last_became_active":"2023-11-28T11:19:44.424531+0000","last_became_peered":"2023-11-28T11:19:44.424531+0000","last_unstale":"2023-11-28T11:19:45.629499+0000","last_undegraded":"2023-11-28T11:19:45.629499+0000","last_fullsized":"2023-11-28T11:19:45.629499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:14.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412704+0000","last_change":"2023-11-28T11:19:44.437722+0000","last_active":"2023-11-28T11:19:45.412704+0000","last_peered":"2023-11-28T11:19:45.412704+0000","last_clean":"2023-11-28T11:19:45.412704+0000","last_became_active":"2023-11-28T11:19:44.437417+0000","last_became_peered":"2023-11-28T11:19:44.437417+0000","last_unstale":"2023-11-28T11:19:45.412704+0000","last_undegraded":"2023-11-28T11:19:45.412704+0000","last_fullsized":"2023-11-28T11:19:45.412704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:31.471220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412521+0000","last_change":"2023-11-28T11:19:44.428328+0000","last_active":"2023-11-28T11:19:45.412521+0000","last_peered":"2023-11-28T11:19:45.412521+0000","last_clean":"2023-11-28T11:19:45.412521+0000","last_became_active":"2023-11-28T11:19:44.428016+0000","last_became_peered":"2023-11-28T11:19:44.428016+0000","last_unstale":"2023-11-28T11:19:45.412521+0000","last_undegraded":"2023-11-28T11:19:45.412521+0000","last_fullsized":"2023-11-28T11:19:45.412521+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:26.387178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.428631+0000","last_change":"2023-11-28T11:19:44.427667+0000","last_active":"2023-11-28T11:19:44.428631+0000","last_peered":"2023-11-28T11:19:44.428631+0000","last_clean":"2023-11-28T11:19:44.428631+0000","last_became_active":"2023-11-28T11:19:44.427446+0000","last_became_peered":"2023-11-28T11:19:44.427446+0000","last_unstale":"2023-11-28T11:19:44.428631+0000","last_undegraded":"2023-11-28T11:19:44.428631+0000","last_fullsized":"2023-11-28T11:19:44.428631+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:25.805157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629092+0000","last_change":"2023-11-28T11:19:44.420659+0000","last_active":"2023-11-28T11:19:45.629092+0000","last_peered":"2023-11-28T11:19:45.629092+0000","last_clean":"2023-11-28T11:19:45.629092+0000","last_became_active":"2023-11-28T11:19:44.420265+0000","last_became_peered":"2023-11-28T11:19:44.420265+0000","last_unstale":"2023-11-28T11:19:45.629092+0000","last_undegraded":"2023-11-28T11:19:45.629092+0000","last_fullsized":"2023-11-28T11:19:45.629092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:39.119998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410896+0000","last_change":"2023-11-28T11:19:44.440246+0000","last_active":"2023-11-28T11:19:45.410896+0000","last_peered":"2023-11-28T11:19:45.410896+0000","last_clean":"2023-11-28T11:19:45.410896+0000","last_became_active":"2023-11-28T11:19:44.439978+0000","last_became_peered":"2023-11-28T11:19:44.439978+0000","last_unstale":"2023-11-28T11:19:45.410896+0000","last_undegraded":"2023-11-28T11:19:45.410896+0000","last_fullsized":"2023-11-28T11:19:45.410896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:30.934789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411377+0000","last_change":"2023-11-28T11:19:44.441037+0000","last_active":"2023-11-28T11:19:45.411377+0000","last_peered":"2023-11-28T11:19:45.411377+0000","last_clean":"2023-11-28T11:19:45.411377+0000","last_became_active":"2023-11-28T11:19:44.440517+0000","last_became_peered":"2023-11-28T11:19:44.440517+0000","last_unstale":"2023-11-28T11:19:45.411377+0000","last_undegraded":"2023-11-28T11:19:45.411377+0000","last_fullsized":"2023-11-28T11:19:45.411377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:51.124532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422371+0000","last_change":"2023-11-28T11:19:44.412751+0000","last_active":"2023-11-28T11:19:44.422371+0000","last_peered":"2023-11-28T11:19:44.422371+0000","last_clean":"2023-11-28T11:19:44.422371+0000","last_became_active":"2023-11-28T11:19:44.412441+0000","last_became_peered":"2023-11-28T11:19:44.412441+0000","last_unstale":"2023-11-28T11:19:44.422371+0000","last_undegraded":"2023-11-28T11:19:44.422371+0000","last_fullsized":"2023-11-28T11:19:44.422371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:07.144404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.426143+0000","last_change":"2023-11-28T11:19:44.422198+0000","last_active":"2023-11-28T11:19:44.426143+0000","last_peered":"2023-11-28T11:19:44.426143+0000","last_clean":"2023-11-28T11:19:44.426143+0000","last_became_active":"2023-11-28T11:19:44.421903+0000","last_became_peered":"2023-11-28T11:19:44.421903+0000","last_unstale":"2023-11-28T11:19:44.426143+0000","last_undegraded":"2023-11-28T11:19:44.426143+0000","last_fullsized":"2023-11-28T11:19:44.426143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:01.437532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411334+0000","last_change":"2023-11-28T11:19:44.434590+0000","last_active":"2023-11-28T11:19:45.411334+0000","last_peered":"2023-11-28T11:19:45.411334+0000","last_clean":"2023-11-28T11:19:45.411334+0000","last_became_active":"2023-11-28T11:19:44.434306+0000","last_became_peered":"2023-11-28T11:19:44.434306+0000","last_unstale":"2023-11-28T11:19:45.411334+0000","last_undegraded":"2023-11-28T11:19:45.411334+0000","last_fullsized":"2023-11-28T11:19:45.411334+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:16:20.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422230+0000","last_change":"2023-11-28T11:19:44.415253+0000","last_active":"2023-11-28T11:19:44.422230+0000","last_peered":"2023-11-28T11:19:44.422230+0000","last_clean":"2023-11-28T11:19:44.422230+0000","last_became_active":"2023-11-28T11:19:44.414979+0000","last_became_peered":"2023-11-28T11:19:44.414979+0000","last_unstale":"2023-11-28T11:19:44.422230+0000","last_undegraded":"2023-11-28T11:19:44.422230+0000","last_fullsized":"2023-11-28T11:19:44.422230+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:31.471220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629252+0000","last_change":"2023-11-28T11:19:44.421005+0000","last_active":"2023-11-28T11:19:45.629252+0000","last_peered":"2023-11-28T11:19:45.629252+0000","last_clean":"2023-11-28T11:19:45.629252+0000","last_became_active":"2023-11-28T11:19:44.420364+0000","last_became_peered":"2023-11-28T11:19:44.420364+0000","last_unstale":"2023-11-28T11:19:45.629252+0000","last_undegraded":"2023-11-28T11:19:45.629252+0000","last_fullsized":"2023-11-28T11:19:45.629252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:39.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412333+0000","last_change":"2023-11-28T11:19:44.433747+0000","last_active":"2023-11-28T11:19:45.412333+0000","last_peered":"2023-11-28T11:19:45.412333+0000","last_clean":"2023-11-28T11:19:45.412333+0000","last_became_active":"2023-11-28T11:19:44.433472+0000","last_became_peered":"2023-11-28T11:19:44.433472+0000","last_unstale":"2023-11-28T11:19:45.412333+0000","last_undegraded":"2023-11-28T11:19:45.412333+0000","last_fullsized":"2023-11-28T11:19:45.412333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:15.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.417992+0000","last_change":"2023-11-28T11:19:44.411732+0000","last_active":"2023-11-28T11:19:44.417992+0000","last_peered":"2023-11-28T11:19:44.417992+0000","last_clean":"2023-11-28T11:19:44.417992+0000","last_became_active":"2023-11-28T11:19:44.411449+0000","last_became_peered":"2023-11-28T11:19:44.411449+0000","last_unstale":"2023-11-28T11:19:44.417992+0000","last_undegraded":"2023-11-28T11:19:44.417992+0000","last_fullsized":"2023-11-28T11:19:44.417992+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:24.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629795+0000","last_change":"2023-11-28T11:19:44.423670+0000","last_active":"2023-11-28T11:19:45.629795+0000","last_peered":"2023-11-28T11:19:45.629795+0000","last_clean":"2023-11-28T11:19:45.629795+0000","last_became_active":"2023-11-28T11:19:44.423059+0000","last_became_peered":"2023-11-28T11:19:44.423059+0000","last_unstale":"2023-11-28T11:19:45.629795+0000","last_undegraded":"2023-11-28T11:19:45.629795+0000","last_fullsized":"2023-11-28T11:19:45.629795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:51.124532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413124+0000","last_change":"2023-11-28T11:19:44.433802+0000","last_active":"2023-11-28T11:19:45.413124+0000","last_peered":"2023-11-28T11:19:45.413124+0000","last_clean":"2023-11-28T11:19:45.413124+0000","last_became_active":"2023-11-28T11:19:44.433105+0000","last_became_peered":"2023-11-28T11:19:44.433105+0000","last_unstale":"2023-11-28T11:19:45.413124+0000","last_undegraded":"2023-11-28T11:19:45.413124+0000","last_fullsized":"2023-11-28T11:19:45.413124+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:52.159199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629687+0000","last_change":"2023-11-28T11:19:44.425389+0000","last_active":"2023-11-28T11:19:45.629687+0000","last_peered":"2023-11-28T11:19:45.629687+0000","last_clean":"2023-11-28T11:19:45.629687+0000","last_became_active":"2023-11-28T11:19:44.424873+0000","last_became_peered":"2023-11-28T11:19:44.424873+0000","last_unstale":"2023-11-28T11:19:45.629687+0000","last_undegraded":"2023-11-28T11:19:45.629687+0000","last_fullsized":"2023-11-28T11:19:45.629687+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.103036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433585+0000","last_change":"2023-11-28T11:19:44.429138+0000","last_active":"2023-11-28T11:19:44.433585+0000","last_peered":"2023-11-28T11:19:44.433585+0000","last_clean":"2023-11-28T11:19:44.433585+0000","last_became_active":"2023-11-28T11:19:44.424325+0000","last_became_peered":"2023-11-28T11:19:44.424325+0000","last_unstale":"2023-11-28T11:19:44.433585+0000","last_undegraded":"2023-11-28T11:19:44.433585+0000","last_fullsized":"2023-11-28T11:19:44.433585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:10.100216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429042+0000","last_change":"2023-11-28T11:19:44.421827+0000","last_active":"2023-11-28T11:19:44.429042+0000","last_peered":"2023-11-28T11:19:44.429042+0000","last_clean":"2023-11-28T11:19:44.429042+0000","last_became_active":"2023-11-28T11:19:44.421595+0000","last_became_peered":"2023-11-28T11:19:44.421595+0000","last_unstale":"2023-11-28T11:19:44.429042+0000","last_undegraded":"2023-11-28T11:19:44.429042+0000","last_fullsized":"2023-11-28T11:19:44.429042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:27.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410982+0000","last_change":"2023-11-28T11:19:44.438916+0000","last_active":"2023-11-28T11:19:45.410982+0000","last_peered":"2023-11-28T11:19:45.410982+0000","last_clean":"2023-11-28T11:19:45.410982+0000","last_became_active":"2023-11-28T11:19:44.436975+0000","last_became_peered":"2023-11-28T11:19:44.436975+0000","last_unstale":"2023-11-28T11:19:45.410982+0000","last_undegraded":"2023-11-28T11:19:45.410982+0000","last_fullsized":"2023-11-28T11:19:45.410982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:14.235872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.427402+0000","last_change":"2023-11-28T11:19:44.425640+0000","last_active":"2023-11-28T11:19:44.427402+0000","last_peered":"2023-11-28T11:19:44.427402+0000","last_clean":"2023-11-28T11:19:44.427402+0000","last_became_active":"2023-11-28T11:19:44.425382+0000","last_became_peered":"2023-11-28T11:19:44.425382+0000","last_unstale":"2023-11-28T11:19:44.427402+0000","last_undegraded":"2023-11-28T11:19:44.427402+0000","last_fullsized":"2023-11-28T11:19:44.427402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:38.591175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629033+0000","last_change":"2023-11-28T11:19:44.418272+0000","last_active":"2023-11-28T11:19:45.629033+0000","last_peered":"2023-11-28T11:19:45.629033+0000","last_clean":"2023-11-28T11:19:45.629033+0000","last_became_active":"2023-11-28T11:19:44.417864+0000","last_became_peered":"2023-11-28T11:19:44.417864+0000","last_unstale":"2023-11-28T11:19:45.629033+0000","last_undegraded":"2023-11-28T11:19:45.629033+0000","last_fullsized":"2023-11-28T11:19:45.629033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629769+0000","last_change":"2023-11-28T11:19:44.434517+0000","last_active":"2023-11-28T11:19:45.629769+0000","last_peered":"2023-11-28T11:19:45.629769+0000","last_clean":"2023-11-28T11:19:45.629769+0000","last_became_active":"2023-11-28T11:19:44.434266+0000","last_became_peered":"2023-11-28T11:19:44.434266+0000","last_unstale":"2023-11-28T11:19:45.629769+0000","last_undegraded":"2023-11-28T11:19:45.629769+0000","last_fullsized":"2023-11-28T11:19:45.629769+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:38.251735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411240+0000","last_change":"2023-11-28T11:19:44.430179+0000","last_active":"2023-11-28T11:19:45.411240+0000","last_peered":"2023-11-28T11:19:45.411240+0000","last_clean":"2023-11-28T11:19:45.411240+0000","last_became_active":"2023-11-28T11:19:44.429912+0000","last_became_peered":"2023-11-28T11:19:44.429912+0000","last_unstale":"2023-11-28T11:19:45.411240+0000","last_undegraded":"2023-11-28T11:19:45.411240+0000","last_fullsized":"2023-11-28T11:19:45.411240+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436098+0000","last_change":"2023-11-28T11:19:44.433285+0000","last_active":"2023-11-28T11:19:44.436098+0000","last_peered":"2023-11-28T11:19:44.436098+0000","last_clean":"2023-11-28T11:19:44.436098+0000","last_became_active":"2023-11-28T11:19:44.432835+0000","last_became_peered":"2023-11-28T11:19:44.432835+0000","last_unstale":"2023-11-28T11:19:44.436098+0000","last_undegraded":"2023-11-28T11:19:44.436098+0000","last_fullsized":"2023-11-28T11:19:44.436098+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:44.822864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.424505+0000","last_change":"2023-11-28T11:19:44.420034+0000","last_active":"2023-11-28T11:19:44.424505+0000","last_peered":"2023-11-28T11:19:44.424505+0000","last_clean":"2023-11-28T11:19:44.424505+0000","last_became_active":"2023-11-28T11:19:44.419747+0000","last_became_peered":"2023-11-28T11:19:44.419747+0000","last_unstale":"2023-11-28T11:19:44.424505+0000","last_undegraded":"2023-11-28T11:19:44.424505+0000","last_fullsized":"2023-11-28T11:19:44.424505+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:04.071637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629305+0000","last_change":"2023-11-28T11:19:44.419852+0000","last_active":"2023-11-28T11:19:45.629305+0000","last_peered":"2023-11-28T11:19:45.629305+0000","last_clean":"2023-11-28T11:19:45.629305+0000","last_became_active":"2023-11-28T11:19:44.419580+0000","last_became_peered":"2023-11-28T11:19:44.419580+0000","last_unstale":"2023-11-28T11:19:45.629305+0000","last_undegraded":"2023-11-28T11:19:45.629305+0000","last_fullsized":"2023-11-28T11:19:45.629305+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:05.409658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412486+0000","last_change":"2023-11-28T11:19:44.431540+0000","last_active":"2023-11-28T11:19:45.412486+0000","last_peered":"2023-11-28T11:19:45.412486+0000","last_clean":"2023-11-28T11:19:45.412486+0000","last_became_active":"2023-11-28T11:19:44.429838+0000","last_became_peered":"2023-11-28T11:19:44.429838+0000","last_unstale":"2023-11-28T11:19:45.412486+0000","last_undegraded":"2023-11-28T11:19:45.412486+0000","last_fullsized":"2023-11-28T11:19:45.412486+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:15.792643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629203+0000","last_change":"2023-11-28T11:19:44.420982+0000","last_active":"2023-11-28T11:19:45.629203+0000","last_peered":"2023-11-28T11:19:45.629203+0000","last_clean":"2023-11-28T11:19:45.629203+0000","last_became_active":"2023-11-28T11:19:44.420691+0000","last_became_peered":"2023-11-28T11:19:44.420691+0000","last_unstale":"2023-11-28T11:19:45.629203+0000","last_undegraded":"2023-11-28T11:19:45.629203+0000","last_fullsized":"2023-11-28T11:19:45.629203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:30.934789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433617+0000","last_change":"2023-11-28T11:19:44.431282+0000","last_active":"2023-11-28T11:19:44.433617+0000","last_peered":"2023-11-28T11:19:44.433617+0000","last_clean":"2023-11-28T11:19:44.433617+0000","last_became_active":"2023-11-28T11:19:44.430979+0000","last_became_peered":"2023-11-28T11:19:44.430979+0000","last_unstale":"2023-11-28T11:19:44.433617+0000","last_undegraded":"2023-11-28T11:19:44.433617+0000","last_fullsized":"2023-11-28T11:19:44.433617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:29.320378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412012+0000","last_change":"2023-11-28T11:19:44.420787+0000","last_active":"2023-11-28T11:19:45.412012+0000","last_peered":"2023-11-28T11:19:45.412012+0000","last_clean":"2023-11-28T11:19:45.412012+0000","last_became_active":"2023-11-28T11:19:44.420491+0000","last_became_peered":"2023-11-28T11:19:44.420491+0000","last_unstale":"2023-11-28T11:19:45.412012+0000","last_undegraded":"2023-11-28T11:19:45.412012+0000","last_fullsized":"2023-11-28T11:19:45.412012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:26.387178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436077+0000","last_change":"2023-11-28T11:19:43.904752+0000","last_active":"2023-11-28T11:19:44.436077+0000","last_peered":"2023-11-28T11:19:44.436077+0000","last_clean":"2023-11-28T11:19:44.436077+0000","last_became_active":"2023-11-28T11:19:43.904340+0000","last_became_peered":"2023-11-28T11:19:43.904340+0000","last_unstale":"2023-11-28T11:19:44.436077+0000","last_undegraded":"2023-11-28T11:19:44.436077+0000","last_fullsized":"2023-11-28T11:19:44.436077+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:07.203302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433112+0000","last_change":"2023-11-28T11:19:44.431312+0000","last_active":"2023-11-28T11:19:44.433112+0000","last_peered":"2023-11-28T11:19:44.433112+0000","last_clean":"2023-11-28T11:19:44.433112+0000","last_became_active":"2023-11-28T11:19:44.428700+0000","last_became_peered":"2023-11-28T11:19:44.428700+0000","last_unstale":"2023-11-28T11:19:44.433112+0000","last_undegraded":"2023-11-28T11:19:44.433112+0000","last_fullsized":"2023-11-28T11:19:44.433112+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:57.924786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.442196+0000","last_change":"2023-11-28T11:19:43.907309+0000","last_active":"2023-11-28T11:19:44.442196+0000","last_peered":"2023-11-28T11:19:44.442196+0000","last_clean":"2023-11-28T11:19:44.442196+0000","last_became_active":"2023-11-28T11:19:43.907091+0000","last_became_peered":"2023-11-28T11:19:43.907091+0000","last_unstale":"2023-11-28T11:19:44.442196+0000","last_undegraded":"2023-11-28T11:19:44.442196+0000","last_fullsized":"2023-11-28T11:19:44.442196+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:55:49.432119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429901+0000","last_change":"2023-11-28T11:19:44.422736+0000","last_active":"2023-11-28T11:19:44.429901+0000","last_peered":"2023-11-28T11:19:44.429901+0000","last_clean":"2023-11-28T11:19:44.429901+0000","last_became_active":"2023-11-28T11:19:44.422423+0000","last_became_peered":"2023-11-28T11:19:44.422423+0000","last_unstale":"2023-11-28T11:19:44.429901+0000","last_undegraded":"2023-11-28T11:19:44.429901+0000","last_fullsized":"2023-11-28T11:19:44.429901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:21.073037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.443441+0000","last_change":"2023-11-28T11:19:43.915537+0000","last_active":"2023-11-28T11:19:44.443441+0000","last_peered":"2023-11-28T11:19:44.443441+0000","last_clean":"2023-11-28T11:19:44.443441+0000","last_became_active":"2023-11-28T11:19:43.914431+0000","last_became_peered":"2023-11-28T11:19:43.914431+0000","last_unstale":"2023-11-28T11:19:44.443441+0000","last_undegraded":"2023-11-28T11:19:44.443441+0000","last_fullsized":"2023-11-28T11:19:44.443441+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:59.706612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410945+0000","last_change":"2023-11-28T11:19:44.432519+0000","last_active":"2023-11-28T11:19:45.410945+0000","last_peered":"2023-11-28T11:19:45.410945+0000","last_clean":"2023-11-28T11:19:45.410945+0000","last_became_active":"2023-11-28T11:19:44.431956+0000","last_became_peered":"2023-11-28T11:19:44.431956+0000","last_unstale":"2023-11-28T11:19:45.410945+0000","last_undegraded":"2023-11-28T11:19:45.410945+0000","last_fullsized":"2023-11-28T11:19:45.410945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:39.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.440659+0000","last_change":"2023-11-28T11:19:43.906143+0000","last_active":"2023-11-28T11:19:44.440659+0000","last_peered":"2023-11-28T11:19:44.440659+0000","last_clean":"2023-11-28T11:19:44.440659+0000","last_became_active":"2023-11-28T11:19:43.905852+0000","last_became_peered":"2023-11-28T11:19:43.905852+0000","last_unstale":"2023-11-28T11:19:44.440659+0000","last_undegraded":"2023-11-28T11:19:44.440659+0000","last_fullsized":"2023-11-28T11:19:44.440659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:40.554063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413088+0000","last_change":"2023-11-28T11:19:44.417691+0000","last_active":"2023-11-28T11:19:45.413088+0000","last_peered":"2023-11-28T11:19:45.413088+0000","last_clean":"2023-11-28T11:19:45.413088+0000","last_became_active":"2023-11-28T11:19:44.417429+0000","last_became_peered":"2023-11-28T11:19:44.417429+0000","last_unstale":"2023-11-28T11:19:45.413088+0000","last_undegraded":"2023-11-28T11:19:45.413088+0000","last_fullsized":"2023-11-28T11:19:45.413088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629308+0000","last_change":"2023-11-28T11:19:43.910881+0000","last_active":"2023-11-28T11:19:45.629308+0000","last_peered":"2023-11-28T11:19:45.629308+0000","last_clean":"2023-11-28T11:19:45.629308+0000","last_became_active":"2023-11-28T11:19:43.910504+0000","last_became_peered":"2023-11-28T11:19:43.910504+0000","last_unstale":"2023-11-28T11:19:45.629308+0000","last_undegraded":"2023-11-28T11:19:45.629308+0000","last_fullsized":"2023-11-28T11:19:45.629308+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:06.139015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412649+0000","last_change":"2023-11-28T11:19:44.437211+0000","last_active":"2023-11-28T11:19:45.412649+0000","last_peered":"2023-11-28T11:19:45.412649+0000","last_clean":"2023-11-28T11:19:45.412649+0000","last_became_active":"2023-11-28T11:19:44.436967+0000","last_became_peered":"2023-11-28T11:19:44.436967+0000","last_unstale":"2023-11-28T11:19:45.412649+0000","last_undegraded":"2023-11-28T11:19:45.412649+0000","last_fullsized":"2023-11-28T11:19:45.412649+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412280+0000","last_change":"2023-11-28T11:19:43.902672+0000","last_active":"2023-11-28T11:19:45.412280+0000","last_peered":"2023-11-28T11:19:45.412280+0000","last_clean":"2023-11-28T11:19:45.412280+0000","last_became_active":"2023-11-28T11:19:43.902370+0000","last_became_peered":"2023-11-28T11:19:43.902370+0000","last_unstale":"2023-11-28T11:19:45.412280+0000","last_undegraded":"2023-11-28T11:19:45.412280+0000","last_fullsized":"2023-11-28T11:19:45.412280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411355+0000","last_change":"2023-11-28T11:19:44.431721+0000","last_active":"2023-11-28T11:19:45.411355+0000","last_peered":"2023-11-28T11:19:45.411355+0000","last_clean":"2023-11-28T11:19:45.411355+0000","last_became_active":"2023-11-28T11:19:44.431423+0000","last_became_peered":"2023-11-28T11:19:44.431423+0000","last_unstale":"2023-11-28T11:19:45.411355+0000","last_undegraded":"2023-11-28T11:19:45.411355+0000","last_fullsized":"2023-11-28T11:19:45.411355+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.413688+0000","last_change":"2023-11-28T11:19:43.909936+0000","last_active":"2023-11-28T11:19:44.413688+0000","last_peered":"2023-11-28T11:19:44.413688+0000","last_clean":"2023-11-28T11:19:44.413688+0000","last_became_active":"2023-11-28T11:19:43.909555+0000","last_became_peered":"2023-11-28T11:19:43.909555+0000","last_unstale":"2023-11-28T11:19:44.413688+0000","last_undegraded":"2023-11-28T11:19:44.413688+0000","last_fullsized":"2023-11-28T11:19:44.413688+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:33.877574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412778+0000","last_change":"2023-11-28T11:19:44.442153+0000","last_active":"2023-11-28T11:19:45.412778+0000","last_peered":"2023-11-28T11:19:45.412778+0000","last_clean":"2023-11-28T11:19:45.412778+0000","last_became_active":"2023-11-28T11:19:44.441884+0000","last_became_peered":"2023-11-28T11:19:44.441884+0000","last_unstale":"2023-11-28T11:19:45.412778+0000","last_undegraded":"2023-11-28T11:19:45.412778+0000","last_fullsized":"2023-11-28T11:19:45.412778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:57:09.714857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629060+0000","last_change":"2023-11-28T11:19:43.904739+0000","last_active":"2023-11-28T11:19:45.629060+0000","last_peered":"2023-11-28T11:19:45.629060+0000","last_clean":"2023-11-28T11:19:45.629060+0000","last_became_active":"2023-11-28T11:19:43.904476+0000","last_became_peered":"2023-11-28T11:19:43.904476+0000","last_unstale":"2023-11-28T11:19:45.629060+0000","last_undegraded":"2023-11-28T11:19:45.629060+0000","last_fullsized":"2023-11-28T11:19:45.629060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:50.176370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412130+0000","last_change":"2023-11-28T11:19:44.427625+0000","last_active":"2023-11-28T11:19:45.412130+0000","last_peered":"2023-11-28T11:19:45.412130+0000","last_clean":"2023-11-28T11:19:45.412130+0000","last_became_active":"2023-11-28T11:19:44.423771+0000","last_became_peered":"2023-11-28T11:19:44.423771+0000","last_unstale":"2023-11-28T11:19:45.412130+0000","last_undegraded":"2023-11-28T11:19:45.412130+0000","last_fullsized":"2023-11-28T11:19:45.412130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:40:10.501924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412920+0000","last_change":"2023-11-28T11:19:43.911818+0000","last_active":"2023-11-28T11:19:45.412920+0000","last_peered":"2023-11-28T11:19:45.412920+0000","last_clean":"2023-11-28T11:19:45.412920+0000","last_became_active":"2023-11-28T11:19:43.911619+0000","last_became_peered":"2023-11-28T11:19:43.911619+0000","last_unstale":"2023-11-28T11:19:45.412920+0000","last_undegraded":"2023-11-28T11:19:45.412920+0000","last_fullsized":"2023-11-28T11:19:45.412920+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:06.823918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.420172+0000","last_change":"2023-11-28T11:19:44.418304+0000","last_active":"2023-11-28T11:19:44.420172+0000","last_peered":"2023-11-28T11:19:44.420172+0000","last_clean":"2023-11-28T11:19:44.420172+0000","last_became_active":"2023-11-28T11:19:44.414877+0000","last_became_peered":"2023-11-28T11:19:44.414877+0000","last_unstale":"2023-11-28T11:19:44.420172+0000","last_undegraded":"2023-11-28T11:19:44.420172+0000","last_fullsized":"2023-11-28T11:19:44.420172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:33.837740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.413074+0000","last_change":"2023-11-28T11:19:43.902678+0000","last_active":"2023-11-28T11:19:45.413074+0000","last_peered":"2023-11-28T11:19:45.413074+0000","last_clean":"2023-11-28T11:19:45.413074+0000","last_became_active":"2023-11-28T11:19:43.902317+0000","last_became_peered":"2023-11-28T11:19:43.902317+0000","last_unstale":"2023-11-28T11:19:45.413074+0000","last_undegraded":"2023-11-28T11:19:45.413074+0000","last_fullsized":"2023-11-28T11:19:45.413074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:24.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629333+0000","last_change":"2023-11-28T11:19:44.420213+0000","last_active":"2023-11-28T11:19:45.629333+0000","last_peered":"2023-11-28T11:19:45.629333+0000","last_clean":"2023-11-28T11:19:45.629333+0000","last_became_active":"2023-11-28T11:19:44.419930+0000","last_became_peered":"2023-11-28T11:19:44.419930+0000","last_unstale":"2023-11-28T11:19:45.629333+0000","last_undegraded":"2023-11-28T11:19:45.629333+0000","last_fullsized":"2023-11-28T11:19:45.629333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:09:01.560420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412679+0000","last_change":"2023-11-28T11:19:43.916715+0000","last_active":"2023-11-28T11:19:45.412679+0000","last_peered":"2023-11-28T11:19:45.412679+0000","last_clean":"2023-11-28T11:19:45.412679+0000","last_became_active":"2023-11-28T11:19:43.916453+0000","last_became_peered":"2023-11-28T11:19:43.916453+0000","last_unstale":"2023-11-28T11:19:45.412679+0000","last_undegraded":"2023-11-28T11:19:45.412679+0000","last_fullsized":"2023-11-28T11:19:45.412679+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:30:20.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412625+0000","last_change":"2023-11-28T11:19:44.420369+0000","last_active":"2023-11-28T11:19:45.412625+0000","last_peered":"2023-11-28T11:19:45.412625+0000","last_clean":"2023-11-28T11:19:45.412625+0000","last_became_active":"2023-11-28T11:19:44.420113+0000","last_became_peered":"2023-11-28T11:19:44.420113+0000","last_unstale":"2023-11-28T11:19:45.412625+0000","last_undegraded":"2023-11-28T11:19:45.412625+0000","last_fullsized":"2023-11-28T11:19:45.412625+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:52.159199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411336+0000","last_change":"2023-11-28T11:19:43.903511+0000","last_active":"2023-11-28T11:19:45.411336+0000","last_peered":"2023-11-28T11:19:45.411336+0000","last_clean":"2023-11-28T11:19:45.411336+0000","last_became_active":"2023-11-28T11:19:43.902881+0000","last_became_peered":"2023-11-28T11:19:43.902881+0000","last_unstale":"2023-11-28T11:19:45.411336+0000","last_undegraded":"2023-11-28T11:19:45.411336+0000","last_fullsized":"2023-11-28T11:19:45.411336+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:31.447467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.424593+0000","last_change":"2023-11-28T11:19:44.420466+0000","last_active":"2023-11-28T11:19:44.424593+0000","last_peered":"2023-11-28T11:19:44.424593+0000","last_clean":"2023-11-28T11:19:44.424593+0000","last_became_active":"2023-11-28T11:19:44.420122+0000","last_became_peered":"2023-11-28T11:19:44.420122+0000","last_unstale":"2023-11-28T11:19:44.424593+0000","last_undegraded":"2023-11-28T11:19:44.424593+0000","last_fullsized":"2023-11-28T11:19:44.424593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:01.014772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629410+0000","last_change":"2023-11-28T11:19:43.908555+0000","last_active":"2023-11-28T11:19:45.629410+0000","last_peered":"2023-11-28T11:19:45.629410+0000","last_clean":"2023-11-28T11:19:45.629410+0000","last_became_active":"2023-11-28T11:19:43.907872+0000","last_became_peered":"2023-11-28T11:19:43.907872+0000","last_unstale":"2023-11-28T11:19:45.629410+0000","last_undegraded":"2023-11-28T11:19:45.629410+0000","last_fullsized":"2023-11-28T11:19:45.629410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629735+0000","last_change":"2023-11-28T11:19:44.431390+0000","last_active":"2023-11-28T11:19:45.629735+0000","last_peered":"2023-11-28T11:19:45.629735+0000","last_clean":"2023-11-28T11:19:45.629735+0000","last_became_active":"2023-11-28T11:19:44.431077+0000","last_became_peered":"2023-11-28T11:19:44.431077+0000","last_unstale":"2023-11-28T11:19:45.629735+0000","last_undegraded":"2023-11-28T11:19:45.629735+0000","last_fullsized":"2023-11-28T11:19:45.629735+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.107723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.415912+0000","last_change":"2023-11-28T11:19:43.914509+0000","last_active":"2023-11-28T11:19:44.415912+0000","last_peered":"2023-11-28T11:19:44.415912+0000","last_clean":"2023-11-28T11:19:44.415912+0000","last_became_active":"2023-11-28T11:19:43.914224+0000","last_became_peered":"2023-11-28T11:19:43.914224+0000","last_unstale":"2023-11-28T11:19:44.415912+0000","last_undegraded":"2023-11-28T11:19:44.415912+0000","last_fullsized":"2023-11-28T11:19:44.415912+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:03.564484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411307+0000","last_change":"2023-11-28T11:19:44.432262+0000","last_active":"2023-11-28T11:19:45.411307+0000","last_peered":"2023-11-28T11:19:45.411307+0000","last_clean":"2023-11-28T11:19:45.411307+0000","last_became_active":"2023-11-28T11:19:44.431980+0000","last_became_peered":"2023-11-28T11:19:44.431980+0000","last_unstale":"2023-11-28T11:19:45.411307+0000","last_undegraded":"2023-11-28T11:19:45.411307+0000","last_fullsized":"2023-11-28T11:19:45.411307+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:13.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412415+0000","last_change":"2023-11-28T11:19:43.911385+0000","last_active":"2023-11-28T11:19:45.412415+0000","last_peered":"2023-11-28T11:19:45.412415+0000","last_clean":"2023-11-28T11:19:45.412415+0000","last_became_active":"2023-11-28T11:19:43.911133+0000","last_became_peered":"2023-11-28T11:19:43.911133+0000","last_unstale":"2023-11-28T11:19:45.412415+0000","last_undegraded":"2023-11-28T11:19:45.412415+0000","last_fullsized":"2023-11-28T11:19:45.412415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:03.066249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629652+0000","last_change":"2023-11-28T11:19:44.423876+0000","last_active":"2023-11-28T11:19:45.629652+0000","last_peered":"2023-11-28T11:19:45.629652+0000","last_clean":"2023-11-28T11:19:45.629652+0000","last_became_active":"2023-11-28T11:19:44.423428+0000","last_became_peered":"2023-11-28T11:19:44.423428+0000","last_unstale":"2023-11-28T11:19:45.629652+0000","last_undegraded":"2023-11-28T11:19:45.629652+0000","last_fullsized":"2023-11-28T11:19:45.629652+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:14.235872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411011+0000","last_change":"2023-11-28T11:19:43.901095+0000","last_active":"2023-11-28T11:19:45.411011+0000","last_peered":"2023-11-28T11:19:45.411011+0000","last_clean":"2023-11-28T11:19:45.411011+0000","last_became_active":"2023-11-28T11:19:43.900868+0000","last_became_peered":"2023-11-28T11:19:43.900868+0000","last_unstale":"2023-11-28T11:19:45.411011+0000","last_undegraded":"2023-11-28T11:19:45.411011+0000","last_fullsized":"2023-11-28T11:19:45.411011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:06.253620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410912+0000","last_change":"2023-11-28T11:19:44.435236+0000","last_active":"2023-11-28T11:19:45.410912+0000","last_peered":"2023-11-28T11:19:45.410912+0000","last_clean":"2023-11-28T11:19:45.410912+0000","last_became_active":"2023-11-28T11:19:44.434959+0000","last_became_peered":"2023-11-28T11:19:44.434959+0000","last_unstale":"2023-11-28T11:19:45.410912+0000","last_undegraded":"2023-11-28T11:19:45.410912+0000","last_fullsized":"2023-11-28T11:19:45.410912+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:25.634003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629658+0000","last_change":"2023-11-28T11:19:43.916369+0000","last_active":"2023-11-28T11:19:45.629658+0000","last_peered":"2023-11-28T11:19:45.629658+0000","last_clean":"2023-11-28T11:19:45.629658+0000","last_became_active":"2023-11-28T11:19:43.916103+0000","last_became_peered":"2023-11-28T11:19:43.916103+0000","last_unstale":"2023-11-28T11:19:45.629658+0000","last_undegraded":"2023-11-28T11:19:45.629658+0000","last_fullsized":"2023-11-28T11:19:45.629658+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:06.253620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429932+0000","last_change":"2023-11-28T11:19:44.427952+0000","last_active":"2023-11-28T11:19:44.429932+0000","last_peered":"2023-11-28T11:19:44.429932+0000","last_clean":"2023-11-28T11:19:44.429932+0000","last_became_active":"2023-11-28T11:19:44.427784+0000","last_became_peered":"2023-11-28T11:19:44.427784+0000","last_unstale":"2023-11-28T11:19:44.429932+0000","last_undegraded":"2023-11-28T11:19:44.429932+0000","last_fullsized":"2023-11-28T11:19:44.429932+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:41.869600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.435875+0000","last_change":"2023-11-28T11:19:43.908328+0000","last_active":"2023-11-28T11:19:44.435875+0000","last_peered":"2023-11-28T11:19:44.435875+0000","last_clean":"2023-11-28T11:19:44.435875+0000","last_became_active":"2023-11-28T11:19:43.908027+0000","last_became_peered":"2023-11-28T11:19:43.908027+0000","last_unstale":"2023-11-28T11:19:44.435875+0000","last_undegraded":"2023-11-28T11:19:44.435875+0000","last_fullsized":"2023-11-28T11:19:44.435875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:13.290188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412171+0000","last_change":"2023-11-28T11:19:44.421452+0000","last_active":"2023-11-28T11:19:45.412171+0000","last_peered":"2023-11-28T11:19:45.412171+0000","last_clean":"2023-11-28T11:19:45.412171+0000","last_became_active":"2023-11-28T11:19:44.421198+0000","last_became_peered":"2023-11-28T11:19:44.421198+0000","last_unstale":"2023-11-28T11:19:45.412171+0000","last_undegraded":"2023-11-28T11:19:45.412171+0000","last_fullsized":"2023-11-28T11:19:45.412171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:12.572610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.436343+0000","last_change":"2023-11-28T11:19:43.915495+0000","last_active":"2023-11-28T11:19:44.436343+0000","last_peered":"2023-11-28T11:19:44.436343+0000","last_clean":"2023-11-28T11:19:44.436343+0000","last_became_active":"2023-11-28T11:19:43.915226+0000","last_became_peered":"2023-11-28T11:19:43.915226+0000","last_unstale":"2023-11-28T11:19:44.436343+0000","last_undegraded":"2023-11-28T11:19:44.436343+0000","last_fullsized":"2023-11-28T11:19:44.436343+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.585787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422212+0000","last_change":"2023-11-28T11:19:44.412664+0000","last_active":"2023-11-28T11:19:44.422212+0000","last_peered":"2023-11-28T11:19:44.422212+0000","last_clean":"2023-11-28T11:19:44.422212+0000","last_became_active":"2023-11-28T11:19:44.412425+0000","last_became_peered":"2023-11-28T11:19:44.412425+0000","last_unstale":"2023-11-28T11:19:44.422212+0000","last_undegraded":"2023-11-28T11:19:44.422212+0000","last_fullsized":"2023-11-28T11:19:44.422212+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:57.310680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.415421+0000","last_change":"2023-11-28T11:19:43.907529+0000","last_active":"2023-11-28T11:19:44.415421+0000","last_peered":"2023-11-28T11:19:44.415421+0000","last_clean":"2023-11-28T11:19:44.415421+0000","last_became_active":"2023-11-28T11:19:43.907238+0000","last_became_peered":"2023-11-28T11:19:43.907238+0000","last_unstale":"2023-11-28T11:19:44.415421+0000","last_undegraded":"2023-11-28T11:19:44.415421+0000","last_fullsized":"2023-11-28T11:19:44.415421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:13.290188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.418211+0000","last_change":"2023-11-28T11:19:44.414611+0000","last_active":"2023-11-28T11:19:44.418211+0000","last_peered":"2023-11-28T11:19:44.418211+0000","last_clean":"2023-11-28T11:19:44.418211+0000","last_became_active":"2023-11-28T11:19:44.414348+0000","last_became_peered":"2023-11-28T11:19:44.414348+0000","last_unstale":"2023-11-28T11:19:44.418211+0000","last_undegraded":"2023-11-28T11:19:44.418211+0000","last_fullsized":"2023-11-28T11:19:44.418211+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.308677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.440307+0000","last_change":"2023-11-28T11:19:43.915646+0000","last_active":"2023-11-28T11:19:44.440307+0000","last_peered":"2023-11-28T11:19:44.440307+0000","last_clean":"2023-11-28T11:19:44.440307+0000","last_became_active":"2023-11-28T11:19:43.915261+0000","last_became_peered":"2023-11-28T11:19:43.915261+0000","last_unstale":"2023-11-28T11:19:44.440307+0000","last_undegraded":"2023-11-28T11:19:44.440307+0000","last_fullsized":"2023-11-28T11:19:44.440307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:55.937899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.420802+0000","last_change":"2023-11-28T11:19:44.418876+0000","last_active":"2023-11-28T11:19:44.420802+0000","last_peered":"2023-11-28T11:19:44.420802+0000","last_clean":"2023-11-28T11:19:44.420802+0000","last_became_active":"2023-11-28T11:19:44.418558+0000","last_became_peered":"2023-11-28T11:19:44.418558+0000","last_unstale":"2023-11-28T11:19:44.420802+0000","last_undegraded":"2023-11-28T11:19:44.420802+0000","last_fullsized":"2023-11-28T11:19:44.420802+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:53.926768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.410994+0000","last_change":"2023-11-28T11:19:43.901092+0000","last_active":"2023-11-28T11:19:45.410994+0000","last_peered":"2023-11-28T11:19:45.410994+0000","last_clean":"2023-11-28T11:19:45.410994+0000","last_became_active":"2023-11-28T11:19:43.900822+0000","last_became_peered":"2023-11-28T11:19:43.900822+0000","last_unstale":"2023-11-28T11:19:45.410994+0000","last_undegraded":"2023-11-28T11:19:45.410994+0000","last_fullsized":"2023-11-28T11:19:45.410994+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629555+0000","last_change":"2023-11-28T11:19:44.426361+0000","last_active":"2023-11-28T11:19:45.629555+0000","last_peered":"2023-11-28T11:19:45.629555+0000","last_clean":"2023-11-28T11:19:45.629555+0000","last_became_active":"2023-11-28T11:19:44.425831+0000","last_became_peered":"2023-11-28T11:19:44.425831+0000","last_unstale":"2023-11-28T11:19:45.629555+0000","last_undegraded":"2023-11-28T11:19:45.629555+0000","last_fullsized":"2023-11-28T11:19:45.629555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:16.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629138+0000","last_change":"2023-11-28T11:19:43.915247+0000","last_active":"2023-11-28T11:19:45.629138+0000","last_peered":"2023-11-28T11:19:45.629138+0000","last_clean":"2023-11-28T11:19:45.629138+0000","last_became_active":"2023-11-28T11:19:43.914898+0000","last_became_peered":"2023-11-28T11:19:43.914898+0000","last_unstale":"2023-11-28T11:19:45.629138+0000","last_undegraded":"2023-11-28T11:19:45.629138+0000","last_fullsized":"2023-11-28T11:19:45.629138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:24.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.430466+0000","last_change":"2023-11-28T11:19:44.429505+0000","last_active":"2023-11-28T11:19:44.430466+0000","last_peered":"2023-11-28T11:19:44.430466+0000","last_clean":"2023-11-28T11:19:44.430466+0000","last_became_active":"2023-11-28T11:19:44.429216+0000","last_became_peered":"2023-11-28T11:19:44.429216+0000","last_unstale":"2023-11-28T11:19:44.430466+0000","last_undegraded":"2023-11-28T11:19:44.430466+0000","last_fullsized":"2023-11-28T11:19:44.430466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:51.181759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.411363+0000","last_change":"2023-11-28T11:19:43.908495+0000","last_active":"2023-11-28T11:19:44.411363+0000","last_peered":"2023-11-28T11:19:44.411363+0000","last_clean":"2023-11-28T11:19:44.411363+0000","last_became_active":"2023-11-28T11:19:43.908190+0000","last_became_peered":"2023-11-28T11:19:43.908190+0000","last_unstale":"2023-11-28T11:19:44.411363+0000","last_undegraded":"2023-11-28T11:19:44.411363+0000","last_fullsized":"2023-11-28T11:19:44.411363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:07.203302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629568+0000","last_change":"2023-11-28T11:19:44.426534+0000","last_active":"2023-11-28T11:19:45.629568+0000","last_peered":"2023-11-28T11:19:45.629568+0000","last_clean":"2023-11-28T11:19:45.629568+0000","last_became_active":"2023-11-28T11:19:44.426116+0000","last_became_peered":"2023-11-28T11:19:44.426116+0000","last_unstale":"2023-11-28T11:19:45.629568+0000","last_undegraded":"2023-11-28T11:19:45.629568+0000","last_fullsized":"2023-11-28T11:19:45.629568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:25:21.326185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412251+0000","last_change":"2023-11-28T11:19:43.908193+0000","last_active":"2023-11-28T11:19:45.412251+0000","last_peered":"2023-11-28T11:19:45.412251+0000","last_clean":"2023-11-28T11:19:45.412251+0000","last_became_active":"2023-11-28T11:19:43.905448+0000","last_became_peered":"2023-11-28T11:19:43.905448+0000","last_unstale":"2023-11-28T11:19:45.412251+0000","last_undegraded":"2023-11-28T11:19:45.412251+0000","last_fullsized":"2023-11-28T11:19:45.412251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.915245+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:19:43.915245+0000","last_peered":"2023-11-28T11:19:43.915245+0000","last_clean":"2023-11-28T11:19:43.915245+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:19:43.915245+0000","last_undegraded":"2023-11-28T11:19:43.915245+0000","last_fullsized":"2023-11-28T11:19:43.915245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.433596+0000","last_change":"2023-11-28T11:19:44.431134+0000","last_active":"2023-11-28T11:19:44.433596+0000","last_peered":"2023-11-28T11:19:44.433596+0000","last_clean":"2023-11-28T11:19:44.433596+0000","last_became_active":"2023-11-28T11:19:44.430858+0000","last_became_peered":"2023-11-28T11:19:44.430858+0000","last_unstale":"2023-11-28T11:19:44.433596+0000","last_undegraded":"2023-11-28T11:19:44.433596+0000","last_fullsized":"2023-11-28T11:19:44.433596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:41.514908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.444709+0000","last_change":"2023-11-28T11:19:43.910068+0000","last_active":"2023-11-28T11:19:44.444709+0000","last_peered":"2023-11-28T11:19:44.444709+0000","last_clean":"2023-11-28T11:19:44.444709+0000","last_became_active":"2023-11-28T11:19:43.909181+0000","last_became_peered":"2023-11-28T11:19:43.909181+0000","last_unstale":"2023-11-28T11:19:44.444709+0000","last_undegraded":"2023-11-28T11:19:44.444709+0000","last_fullsized":"2023-11-28T11:19:44.444709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:54.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.900652+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:19:43.900652+0000","last_peered":"2023-11-28T11:19:43.900652+0000","last_clean":"2023-11-28T11:19:43.900652+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:19:43.900652+0000","last_undegraded":"2023-11-28T11:19:43.900652+0000","last_fullsized":"2023-11-28T11:19:43.900652+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411069+0000","last_change":"2023-11-28T11:19:44.432001+0000","last_active":"2023-11-28T11:19:45.411069+0000","last_peered":"2023-11-28T11:19:45.411069+0000","last_clean":"2023-11-28T11:19:45.411069+0000","last_became_active":"2023-11-28T11:19:44.431723+0000","last_became_peered":"2023-11-28T11:19:44.431723+0000","last_unstale":"2023-11-28T11:19:45.411069+0000","last_undegraded":"2023-11-28T11:19:45.411069+0000","last_fullsized":"2023-11-28T11:19:45.411069+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:30.493421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629501+0000","last_change":"2023-11-28T11:19:43.910130+0000","last_active":"2023-11-28T11:19:45.629501+0000","last_peered":"2023-11-28T11:19:45.629501+0000","last_clean":"2023-11-28T11:19:45.629501+0000","last_became_active":"2023-11-28T11:19:43.907373+0000","last_became_peered":"2023-11-28T11:19:43.907373+0000","last_unstale":"2023-11-28T11:19:45.629501+0000","last_undegraded":"2023-11-28T11:19:45.629501+0000","last_fullsized":"2023-11-28T11:19:45.629501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.892135+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:19:43.892135+0000","last_peered":"2023-11-28T11:19:43.892135+0000","last_clean":"2023-11-28T11:19:43.892135+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:19:43.892135+0000","last_undegraded":"2023-11-28T11:19:43.892135+0000","last_fullsized":"2023-11-28T11:19:43.892135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.429021+0000","last_change":"2023-11-28T11:19:44.424224+0000","last_active":"2023-11-28T11:19:44.429021+0000","last_peered":"2023-11-28T11:19:44.429021+0000","last_clean":"2023-11-28T11:19:44.429021+0000","last_became_active":"2023-11-28T11:19:44.423861+0000","last_became_peered":"2023-11-28T11:19:44.423861+0000","last_unstale":"2023-11-28T11:19:44.429021+0000","last_undegraded":"2023-11-28T11:19:44.429021+0000","last_fullsized":"2023-11-28T11:19:44.429021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:48.741292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629626+0000","last_change":"2023-11-28T11:19:43.909558+0000","last_active":"2023-11-28T11:19:45.629626+0000","last_peered":"2023-11-28T11:19:45.629626+0000","last_clean":"2023-11-28T11:19:45.629626+0000","last_became_active":"2023-11-28T11:19:43.909307+0000","last_became_peered":"2023-11-28T11:19:43.909307+0000","last_unstale":"2023-11-28T11:19:45.629626+0000","last_undegraded":"2023-11-28T11:19:45.629626+0000","last_fullsized":"2023-11-28T11:19:45.629626+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:16.586825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.900878+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:19:43.900878+0000","last_peered":"2023-11-28T11:19:43.900878+0000","last_clean":"2023-11-28T11:19:43.900878+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:19:43.900878+0000","last_undegraded":"2023-11-28T11:19:43.900878+0000","last_fullsized":"2023-11-28T11:19:43.900878+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412139+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:19:45.412139+0000","last_peered":"2023-11-28T11:19:45.412139+0000","last_clean":"2023-11-28T11:19:45.412139+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:19:45.412139+0000","last_undegraded":"2023-11-28T11:19:45.412139+0000","last_fullsized":"2023-11-28T11:19:45.412139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.421640+0000","last_change":"2023-11-28T11:19:44.418894+0000","last_active":"2023-11-28T11:19:44.421640+0000","last_peered":"2023-11-28T11:19:44.421640+0000","last_clean":"2023-11-28T11:19:44.421640+0000","last_became_active":"2023-11-28T11:19:44.418485+0000","last_became_peered":"2023-11-28T11:19:44.418485+0000","last_unstale":"2023-11-28T11:19:44.421640+0000","last_undegraded":"2023-11-28T11:19:44.421640+0000","last_fullsized":"2023-11-28T11:19:44.421640+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:32.452856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412820+0000","last_change":"2023-11-28T11:19:43.911892+0000","last_active":"2023-11-28T11:19:45.412820+0000","last_peered":"2023-11-28T11:19:45.412820+0000","last_clean":"2023-11-28T11:19:45.412820+0000","last_became_active":"2023-11-28T11:19:43.911116+0000","last_became_peered":"2023-11-28T11:19:43.911116+0000","last_unstale":"2023-11-28T11:19:45.412820+0000","last_undegraded":"2023-11-28T11:19:45.412820+0000","last_fullsized":"2023-11-28T11:19:45.412820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:23.854989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.412212+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:19:45.412212+0000","last_peered":"2023-11-28T11:19:45.412212+0000","last_clean":"2023-11-28T11:19:45.412212+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:19:45.412212+0000","last_undegraded":"2023-11-28T11:19:45.412212+0000","last_fullsized":"2023-11-28T11:19:45.412212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411402+0000","last_change":"2023-11-28T11:19:44.429036+0000","last_active":"2023-11-28T11:19:45.411402+0000","last_peered":"2023-11-28T11:19:45.411402+0000","last_clean":"2023-11-28T11:19:45.411402+0000","last_became_active":"2023-11-28T11:19:44.428601+0000","last_became_peered":"2023-11-28T11:19:44.428601+0000","last_unstale":"2023-11-28T11:19:45.411402+0000","last_undegraded":"2023-11-28T11:19:45.411402+0000","last_fullsized":"2023-11-28T11:19:45.411402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:26.544984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.410563+0000","last_change":"2023-11-28T11:19:43.913644+0000","last_active":"2023-11-28T11:19:44.410563+0000","last_peered":"2023-11-28T11:19:44.410563+0000","last_clean":"2023-11-28T11:19:44.410563+0000","last_became_active":"2023-11-28T11:19:43.913370+0000","last_became_peered":"2023-11-28T11:19:43.913370+0000","last_unstale":"2023-11-28T11:19:44.410563+0000","last_undegraded":"2023-11-28T11:19:44.410563+0000","last_fullsized":"2023-11-28T11:19:44.410563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:44.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.422285+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:19:44.422285+0000","last_peered":"2023-11-28T11:19:44.422285+0000","last_clean":"2023-11-28T11:19:44.422285+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:19:44.422285+0000","last_undegraded":"2023-11-28T11:19:44.422285+0000","last_fullsized":"2023-11-28T11:19:44.422285+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.629408+0000","last_change":"2023-11-28T11:19:44.443917+0000","last_active":"2023-11-28T11:19:45.629408+0000","last_peered":"2023-11-28T11:19:45.629408+0000","last_clean":"2023-11-28T11:19:45.629408+0000","last_became_active":"2023-11-28T11:19:44.440839+0000","last_became_peered":"2023-11-28T11:19:44.440839+0000","last_unstale":"2023-11-28T11:19:45.629408+0000","last_undegraded":"2023-11-28T11:19:45.629408+0000","last_fullsized":"2023-11-28T11:19:45.629408+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:38.183285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411103+0000","last_change":"2023-11-28T11:19:43.902283+0000","last_active":"2023-11-28T11:19:45.411103+0000","last_peered":"2023-11-28T11:19:45.411103+0000","last_clean":"2023-11-28T11:19:45.411103+0000","last_became_active":"2023-11-28T11:19:43.901683+0000","last_became_peered":"2023-11-28T11:19:43.901683+0000","last_unstale":"2023-11-28T11:19:45.411103+0000","last_undegraded":"2023-11-28T11:19:45.411103+0000","last_fullsized":"2023-11-28T11:19:45.411103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:50.176370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.915355+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:19:43.915355+0000","last_peered":"2023-11-28T11:19:43.915355+0000","last_clean":"2023-11-28T11:19:43.915355+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:19:43.915355+0000","last_undegraded":"2023-11-28T11:19:43.915355+0000","last_fullsized":"2023-11-28T11:19:43.915355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T11:19:44.432126+0000","last_change":"2023-11-28T11:19:44.429559+0000","last_active":"2023-11-28T11:19:44.432126+0000","last_peered":"2023-11-28T11:19:44.432126+0000","last_clean":"2023-11-28T11:19:44.432126+0000","last_became_active":"2023-11-28T11:19:44.429301+0000","last_became_peered":"2023-11-28T11:19:44.429301+0000","last_unstale":"2023-11-28T11:19:44.432126+0000","last_undegraded":"2023-11-28T11:19:44.432126+0000","last_fullsized":"2023-11-28T11:19:44.432126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:43.881874+0000","last_clean_scrub_stamp":"2023-11-28T11:19:43.881874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:59.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:19:45.411078+0000","last_change":"2023-11-28T11:19:43.910348+0000","last_active":"2023-11-28T11:19:45.411078+0000","last_peered":"2023-11-28T11:19:45.411078+0000","last_clean":"2023-11-28T11:19:45.411078+0000","last_became_active":"2023-11-28T11:19:43.910058+0000","last_became_peered":"2023-11-28T11:19:43.910058+0000","last_unstale":"2023-11-28T11:19:45.411078+0000","last_undegraded":"2023-11-28T11:19:45.411078+0000","last_fullsized":"2023-11-28T11:19:45.411078+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:19:42.876485+0000","last_clean_scrub_stamp":"2023-11-28T11:19:42.876485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:56.305291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:19:43.897929+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:19:43.897929+0000","last_peered":"2023-11-28T11:19:43.897929+0000","last_clean":"2023-11-28T11:19:43.897929+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:19:43.897929+0000","last_undegraded":"2023-11-28T11:19:43.897929+0000","last_fullsized":"2023-11-28T11:19:43.897929+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14264,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357576,"statfs":{"total":96636764160,"available":96622157824,"internally_reserved":0,"allocated":1163264,"data_stored":831812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78700000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66100000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88400000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.69299999999999995},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.92800000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96799999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97699999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.747},{"interface":"front","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53700000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12904,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358936,"statfs":{"total":96636764160,"available":96623550464,"internally_reserved":0,"allocated":557056,"data_stored":235303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.53500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":2.23,"15min":2.23},"last":0.84199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79300000000000004,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83999999999999997,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94199999999999995,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.56699999999999995,"5min":0.499,"15min":0.499},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82399999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90400000000000003,"5min":1.627,"15min":1.627},"last":0.88}]},{"osd":4,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96599999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90700000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.55000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12964,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94358876,"statfs":{"total":96636764160,"available":96623489024,"internally_reserved":0,"allocated":552960,"data_stored":233863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56100000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.495},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46400000000000002,"15min":0.46700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95999999999999996,"5min":1.181,"15min":1.181},"last":0.83099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.57599999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86299999999999999,"5min":1.774,"15min":1.774},"last":0.86299999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.753},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97899999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.052,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90700000000000003,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.90200000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14120,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94357720,"statfs":{"total":96636764160,"available":96622305280,"internally_reserved":0,"allocated":1146880,"data_stored":824072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61199999999999999,"5min":1.052,"15min":1.052},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69199999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61199999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.056,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.629},{"interface":"front","average":{"1min":0.754,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.163,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.022,"5min":1.458,"15min":1.458},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.98299999999999998,"5min":1.026,"15min":1.026},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.58899999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.129,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.755},{"interface":"front","average":{"1min":0.745,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1419999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12908,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358932,"statfs":{"total":96636764160,"available":96623546368,"internally_reserved":0,"allocated":561152,"data_stored":236215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.42399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.4580000000000002,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.63100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13244,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358596,"statfs":{"total":96636764160,"available":96623202304,"internally_reserved":0,"allocated":577536,"data_stored":242890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.625,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47099999999999997,"15min":0.47699999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66600000000000004,"5min":3.004,"15min":3.004},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.628,"15min":0.65200000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69299999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":3.274,"15min":3.274},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.52900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.095,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.91800000000000004,"15min":0.94599999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.014,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.81999999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.91900000000000004,"15min":0.94899999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0289999999999999,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0649999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.622,"15min":0.622},"max":{"1min":1.004,"5min":1.099,"15min":1.099},"last":0.88400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13232,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358608,"statfs":{"total":96636764160,"available":96623214592,"internally_reserved":0,"allocated":565248,"data_stored":241886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.56200000000000006}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3720000000000001,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.877,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.002,"5min":1.847,"15min":1.847},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":1.615,"15min":1.615},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.127,"15min":1.127},"last":0.72899999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.55000000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0329999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97899999999999998,"5min":1.296,"15min":1.296},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93799999999999994,"5min":1.212,"15min":1.212},"last":0.503}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13088,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358752,"statfs":{"total":96636764160,"available":96623362048,"internally_reserved":0,"allocated":548864,"data_stored":234667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.68899999999999995,"15min":0.66500000000000004},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72499999999999998,"15min":0.73599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.367,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.442}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.3199999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":2.661,"15min":2.661},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.375,"5min":5.375,"15min":5.375},"last":0.624},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.73599999999999999,"15min":0.72499999999999998},"min":{"1min":0.439,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.76200000000000001,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.333,"15min":1.333},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.1859999999999999,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:19:49.311 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:19:49.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:19:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.556+0000 7fd0457ff700 1 Processor -- start 2023-11-28T11:19:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.556+0000 7fd0457ff700 1 -- start start 2023-11-28T11:19:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd0457ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd0400cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd0457ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400d1950 con 0x7fd04015c9b0 2023-11-28T11:19:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd0457ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400ce140 con 0x7fd040161340 2023-11-28T11:19:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd0457ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400b5800 con 0x7fd0400cd7c0 2023-11-28T11:19:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd03effd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd04015c9b0 0x7fd0400b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56182/0 (socket says 172.21.15.53:56182) 2023-11-28T11:19:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.557+0000 7fd03effd700 1 -- 172.21.15.53:0/2816984315 learned_addr learned my addr 172.21.15.53:0/2816984315 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583281986 0 0) 0x7fd0400d1950 con 0x7fd04015c9b0 2023-11-28T11:19:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0280035b0 con 0x7fd04015c9b0 2023-11-28T11:19:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd0400cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 789959908 0 0) 0x7fd0280035b0 con 0x7fd04015c9b0 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 msgr2=0x7fd0400cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd0400cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 >> v1:172.21.15.157:6789/0 conn(0x7fd040161340 legacy=0x7fd0400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400d24f0 con 0x7fd04015c9b0 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fd030001670 con 0x7fd04015c9b0 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd0400cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 >> v1:172.21.15.53:6789/0 conn(0x7fd04015c9b0 legacy=0x7fd0400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd028003a30 0x7fd028003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd028004850 0x7fd028004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd028005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400b5800 con 0x7fd028003a30 2023-11-28T11:19:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800bbd0 con 0x7fd028004850 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.558+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800bd90 con 0x7fd0400cd7c0 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd028003a30 0x7fd028003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd028005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03e7fc700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd028004850 0x7fd028004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 -- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 msgr2=0x7fd028005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0400cd7c0 0x7fd028005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 -- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd028004850 msgr2=0x7fd028004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd028004850 0x7fd028004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 -- 172.21.15.53:0/2816984315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0280035b0 con 0x7fd028003a30 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.559+0000 7fd03effd700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd028003a30 0x7fd028003e40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c240 tx=0x7fd028005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49ba00db81ab61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030007c10 con 0x7fd028003a30 2023-11-28T11:19:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03000ce00 con 0x7fd028003a30 2023-11-28T11:19:49.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd03dffb700 1 -- 172.21.15.53:0/2816984315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030005ba0 con 0x7fd028003a30 2023-11-28T11:19:49.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 -- 172.21.15.53:0/2816984315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd028003a30 msgr2=0x7fd028003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 --2- 172.21.15.53:0/2816984315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd028003a30 0x7fd028003e40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c240 tx=0x7fd028005880 comp rx=0 tx=0).stop 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd03f7fe700 1 -- 172.21.15.53:0/2816984315 reap_dead start 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 -- 172.21.15.53:0/2816984315 shutdown_connections 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 -- 172.21.15.53:0/2816984315 >> 172.21.15.53:0/2816984315 conn(0x7fd040072600 msgr2=0x7fd0400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 -- 172.21.15.53:0/2816984315 shutdown_connections 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.560+0000 7fd0457ff700 1 -- 172.21.15.53:0/2816984315 wait complete. 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.563+0000 7fd0457ff700 1 Processor -- start 2023-11-28T11:19:49.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.563+0000 7fd0457ff700 1 -- start start 2023-11-28T11:19:49.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd028003a30 0x7fd040157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0400b40e0 0x7fd0400c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04015fe10 con 0x7fd0400b4550 2023-11-28T11:19:49.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04015f190 con 0x7fd0400b40e0 2023-11-28T11:19:49.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.564+0000 7fd0457ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04015f470 con 0x7fd028003a30 2023-11-28T11:19:49.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59550/0 (socket says 172.21.15.53:59550) 2023-11-28T11:19:49.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 -- 172.21.15.53:0/3478601696 learned_addr learned my addr 172.21.15.53:0/3478601696 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:49.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 -- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd028003a30 msgr2=0x7fd040157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd028003a30 0x7fd040157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 -- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0400b40e0 msgr2=0x7fd0400c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03e7fc700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0400b40e0 0x7fd0400c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0400b40e0 0x7fd0400c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 -- 172.21.15.53:0/3478601696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400d1300 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd03f7fe700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d2960 tx=0x7fd038010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03801b070 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038010b30 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038010ce0 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.565+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0400b5520 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.566+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0400d15e0 con 0x7fd0400b4550 2023-11-28T11:19:49.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.567+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0400745c0 con 0x7fd0400b4550 2023-11-28T11:19:49.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.568+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd038017430 con 0x7fd0400b4550 2023-11-28T11:19:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.569+0000 7fd0277fe700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 0x7fd02c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.569+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd038086160 con 0x7fd0400b4550 2023-11-28T11:19:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.569+0000 7fd03effd700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 0x7fd02c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.570+0000 7fd03effd700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 0x7fd02c06a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b5a30 tx=0x7fd03000a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.573+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd038052870 con 0x7fd0400b4550 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.857+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fd0400d0870 con 0x7fd0400b4550 2023-11-28T11:19:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.861+0000 7fd0277fe700 1 -- 172.21.15.53:0/3478601696 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd038055e90 con 0x7fd0400b4550 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 msgr2=0x7fd02c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 0x7fd02c06a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b5a30 tx=0x7fd03000a000 comp rx=0 tx=0).stop 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 msgr2=0x7fd0400c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d2960 tx=0x7fd038010440 comp rx=0 tx=0).stop 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 shutdown_connections 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd02c0682f0 0x7fd02c06a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd028003a30 0x7fd040157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0400b4550 0x7fd0400c7300 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 --2- 172.21.15.53:0/3478601696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0400b40e0 0x7fd0400c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 >> 172.21.15.53:0/3478601696 conn(0x7fd040072600 msgr2=0x7fd0400b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 shutdown_connections 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:49.864+0000 7fd0457ff700 1 -- 172.21.15.53:0/3478601696 wait complete. 2023-11-28T11:19:49.866 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-11-28T11:19:49.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.084+0000 7ff7871ff700 1 Processor -- start 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.084+0000 7ff7871ff700 1 -- start start 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.085+0000 7ff7871ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff7800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.085+0000 7ff7871ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800d1950 con 0x7ff7800cd7c0 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.085+0000 7ff7871ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800d24f0 con 0x7ff780161340 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.085+0000 7ff7871ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800b5800 con 0x7ff78015c9b0 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff78579c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff7800cd7c0 0x7ff7800b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56196/0 (socket says 172.21.15.53:56196) 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff78579c700 1 -- 172.21.15.53:0/1280237458 learned_addr learned my addr 172.21.15.53:0/1280237458 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff784f9b700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff7800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915137656 0 0) 0x7ff7800d1950 con 0x7ff7800cd7c0 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7700035b0 con 0x7ff7800cd7c0 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743137759 0 0) 0x7ff7800d24f0 con 0x7ff780161340 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7800d1950 con 0x7ff780161340 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920193686 0 0) 0x7ff7700035b0 con 0x7ff7800cd7c0 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 msgr2=0x7ff7800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff7800b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.086+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 >> v1:172.21.15.157:6789/0 conn(0x7ff780161340 legacy=0x7ff7800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800b5520 con 0x7ff7800cd7c0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7ff7780019f0 con 0x7ff7800cd7c0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 >> v1:172.21.15.53:6789/0 conn(0x7ff7800cd7c0 legacy=0x7ff7800b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770003bc0 0x7ff770003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff770004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff770004d90 0x7ff770005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800b5800 con 0x7ff770004d90 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bba0 con 0x7ff770003bc0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bd10 con 0x7ff78015c9b0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff770004d90 0x7ff770005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff77ffff700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff770004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff784f9b700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770003bc0 0x7ff770003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 -- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 msgr2=0x7ff770004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff78015c9b0 0x7ff770004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 -- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770003bc0 msgr2=0x7ff770003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770003bc0 0x7ff770003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.087+0000 7ff78579c700 1 -- 172.21.15.53:0/1280237458 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700035b0 con 0x7ff770004d90 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff78579c700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff770004d90 0x7ff770005200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff778002230 tx=0x7ff770005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56e6ed394a59ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7780072d0 con 0x7ff770004d90 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778007430 con 0x7ff770004d90 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff77f7fe700 1 -- 172.21.15.53:0/1280237458 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7780077e0 con 0x7ff770004d90 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff7871ff700 1 -- 172.21.15.53:0/1280237458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff770004d90 msgr2=0x7ff770005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1280237458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff770004d90 0x7ff770005200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff778002230 tx=0x7ff770005880 comp rx=0 tx=0).stop 2023-11-28T11:19:50.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.088+0000 7ff78579c700 1 -- 172.21.15.53:0/1280237458 reap_dead start 2023-11-28T11:19:50.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.089+0000 7ff7871ff700 1 -- 172.21.15.53:0/1280237458 shutdown_connections 2023-11-28T11:19:50.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.089+0000 7ff7871ff700 1 -- 172.21.15.53:0/1280237458 >> 172.21.15.53:0/1280237458 conn(0x7ff780072600 msgr2=0x7ff7800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:50.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.089+0000 7ff7871ff700 1 -- 172.21.15.53:0/1280237458 shutdown_connections 2023-11-28T11:19:50.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.089+0000 7ff7871ff700 1 -- 172.21.15.53:0/1280237458 wait complete. 2023-11-28T11:19:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.091+0000 7ff7871ff700 1 Processor -- start 2023-11-28T11:19:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.091+0000 7ff7871ff700 1 -- start start 2023-11-28T11:19:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.092+0000 7ff7871ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770004d90 0x7ff78015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.092+0000 7ff7871ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800cc6f0 0x7ff7800ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff7871ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff7871ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800cd0a0 con 0x7ff7800ca400 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff7871ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780146260 con 0x7ff770004d90 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff7871ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff780146540 con 0x7ff7800cc6f0 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59572/0 (socket says 172.21.15.53:59572) 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 -- 172.21.15.53:0/1145971572 learned_addr learned my addr 172.21.15.53:0/1145971572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff784f9b700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770004d90 0x7ff78015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff77ffff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800cc6f0 0x7ff7800ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 -- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800cc6f0 msgr2=0x7ff7800ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800cc6f0 0x7ff7800ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 -- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770004d90 msgr2=0x7ff78015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770004d90 0x7ff78015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 -- 172.21.15.53:0/1145971572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800d15e0 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.093+0000 7ff78579c700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff778011000 tx=0x7ff778001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.094+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778025030 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.094+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7780079f0 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.094+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7800d1300 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.094+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778012640 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.094+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78015cd80 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.095+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780147020 con 0x7ff7800ca400 2023-11-28T11:19:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.096+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff778010070 con 0x7ff7800ca400 2023-11-28T11:19:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.097+0000 7ff77d7fa700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 0x7ff76406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.097+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff778026070 con 0x7ff7800ca400 2023-11-28T11:19:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.098+0000 7ff784f9b700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 0x7ff76406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.098+0000 7ff784f9b700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 0x7ff76406a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff774009c60 tx=0x7ff774009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:50.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.101+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff780147020 con 0x7ff7800ca400 2023-11-28T11:19:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.350+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff780146e50 con 0x7ff7800ca400 2023-11-28T11:19:50.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.351+0000 7ff77d7fa700 1 -- 172.21.15.53:0/1145971572 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ff778018020 con 0x7ff7800ca400 2023-11-28T11:19:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:50.352 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:19:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 msgr2=0x7ff76406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 0x7ff76406a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff774009c60 tx=0x7ff774009210 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 msgr2=0x7ff7800ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff778011000 tx=0x7ff778001cb0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 shutdown_connections 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff7640682f0 0x7ff76406a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800ca400 0x7ff7800ca830 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff770004d90 0x7ff78015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 --2- 172.21.15.53:0/1145971572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800cc6f0 0x7ff7800ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 >> 172.21.15.53:0/1145971572 conn(0x7ff780072600 msgr2=0x7ff7800b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 shutdown_connections 2023-11-28T11:19:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.353+0000 7ff7871ff700 1 -- 172.21.15.53:0/1145971572 wait complete. 2023-11-28T11:19:50.356 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:19:50.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:19:44.401098+0000', 'modified': '2023-11-28T11:19:47.410724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55007': {'gid': 55007, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/4065027310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 4065027310}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 4065027310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:19:50.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:19:50.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:19:50.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.980+0000 7f60059ff700 1 Processor -- start 2023-11-28T11:19:50.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.981+0000 7f60059ff700 1 -- start start 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f60059ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f600015c9b0 0x7f60000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f60059ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6000161340 0x7f60000d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f60059ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60000d1950 con 0x7f60000d2320 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f60059ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000b5800 con 0x7f600015c9b0 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f60059ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000b5520 con 0x7f6000161340 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.982+0000 7f5ffeffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f600015c9b0 0x7f60000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5fff7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f60000d2320 0x7f60000cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34198/0 (socket says 172.21.15.53:34198) 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5fff7fe700 1 -- 172.21.15.53:0/763151276 learned_addr learned my addr 172.21.15.53:0/763151276 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:50.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6000161340 0x7f60000d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ffeffd700 1 -- 172.21.15.53:0/763151276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6000161340 msgr2=0x7f60000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6000161340 0x7f60000d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ffeffd700 1 -- 172.21.15.53:0/763151276 >> v1:172.21.15.53:6789/0 conn(0x7f60000d2320 legacy=0x7f60000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ffeffd700 1 -- 172.21.15.53:0/763151276 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000d1300 con 0x7f600015c9b0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.983+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f600015c9b0 0x7f60000b4bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90233931b0753c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f5ffe7fc700 1 -- 172.21.15.53:0/763151276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011830 con 0x7f600015c9b0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f5ffe7fc700 1 -- 172.21.15.53:0/763151276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011990 con 0x7f600015c9b0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f5ffe7fc700 1 -- 172.21.15.53:0/763151276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011d20 con 0x7f600015c9b0 2023-11-28T11:19:50.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 -- 172.21.15.53:0/763151276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f600015c9b0 msgr2=0x7f60000b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f600015c9b0 0x7f60000b4bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).stop 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 -- 172.21.15.53:0/763151276 shutdown_connections 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f600015c9b0 0x7f60000b4bf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 --2- 172.21.15.53:0/763151276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6000161340 0x7f60000d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 -- 172.21.15.53:0/763151276 >> 172.21.15.53:0/763151276 conn(0x7f6000072600 msgr2=0x7f60000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 -- 172.21.15.53:0/763151276 shutdown_connections 2023-11-28T11:19:50.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.984+0000 7f60059ff700 1 -- 172.21.15.53:0/763151276 wait complete. 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.986+0000 7f60059ff700 1 Processor -- start 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.986+0000 7f60059ff700 1 -- start start 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60000d2320 0x7f6000151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 0x7f6000150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000150760 con 0x7f6000161340 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000154af0 con 0x7f600015c9b0 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f60059ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6000154dd0 con 0x7f60000d2320 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35736/0 (socket says 172.21.15.53:35736) 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f5fff7fe700 1 -- 172.21.15.53:0/1715680839 learned_addr learned my addr 172.21.15.53:0/1715680839 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:50.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.987+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60000d2320 0x7f6000151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 0x7f6000150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 -- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60000d2320 msgr2=0x7f6000151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60000d2320 0x7f6000151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 -- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 msgr2=0x7f6000150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 0x7f6000150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:50.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 -- 172.21.15.53:0/1715680839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00094a0 con 0x7f6000161340 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 0x7f6000150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800ca80 tx=0x7f5ff800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5ffcff9700 1 -- 172.21.15.53:0/1715680839 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800a9e0 con 0x7f6000161340 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5ffcff9700 1 -- 172.21.15.53:0/1715680839 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff800ab40 con 0x7f6000161340 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60000b5130 con 0x7f6000161340 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f5ffcff9700 1 -- 172.21.15.53:0/1715680839 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8012400 con 0x7f6000161340 2023-11-28T11:19:50.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.988+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60000d15e0 con 0x7f6000161340 2023-11-28T11:19:50.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.990+0000 7f5ffcff9700 1 -- 172.21.15.53:0/1715680839 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ff8019030 con 0x7f6000161340 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.992+0000 7f5ffcff9700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 0x7f5fe006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.992+0000 7f5ffcff9700 1 -- 172.21.15.53:0/1715680839 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ff8085720 con 0x7f6000161340 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.992+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 0x7f5fe006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.993+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 0x7f5fe006a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.994+0000 7f60192d3b80 1 Processor -- start 2023-11-28T11:19:51.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.994+0000 7f60192d3b80 1 -- start start 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0cdfda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cce7bc0 con 0x55bb0ce25130 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cd56350 con 0x55bb0ce27810 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cd4f880 con 0x55bb0ce28120 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35742/0 (socket says 172.21.15.53:35742) 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f5ff7fff700 1 -- 172.21.15.53:0/2080146569 learned_addr learned my addr 172.21.15.53:0/2080146569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0cdfda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.995+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 -- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 msgr2=0x55bb0ce43770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce43770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 -- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 msgr2=0x55bb0cdfda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0cdfda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 -- 172.21.15.53:0/2080146569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb0cbd15e0 con 0x55bb0ce25130 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009cb0 tx=0x7f5fe8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28b935988c228b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff57fa700 1 -- 172.21.15.53:0/2080146569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8011a70 con 0x55bb0ce25130 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff57fa700 1 -- 172.21.15.53:0/2080146569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8011bd0 con 0x55bb0ce25130 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.996+0000 7f5ff57fa700 1 -- 172.21.15.53:0/2080146569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8015460 con 0x55bb0ce25130 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 -- 172.21.15.53:0/2080146569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 msgr2=0x55bb0ce27be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009cb0 tx=0x7f5fe8009210 comp rx=0 tx=0).stop 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 -- 172.21.15.53:0/2080146569 shutdown_connections 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce43770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0cdfda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2080146569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce27be0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 -- 172.21.15.53:0/2080146569 >> 172.21.15.53:0/2080146569 conn(0x55bb0cdfad60 msgr2=0x55bb0cdfb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 -- 172.21.15.53:0/2080146569 shutdown_connections 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.997+0000 7f60192d3b80 1 -- 172.21.15.53:0/2080146569 wait complete. 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.998+0000 7f60192d3b80 1 -- start start 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.998+0000 7f60192d3b80 1 Processor -- start 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce27810 0x55bb0ce1e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cbd38d0 con 0x55bb0ce25130 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0ce47f20 con 0x55bb0ce28120 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb0ce4a730 con 0x55bb0ce27810 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f60192d3b80 10 client.0 _ll_register_callbacks cb 0x55bb0cbf5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:51.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35756/0 (socket says 172.21.15.53:35756) 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:50.999+0000 7f5ffeffd700 1 -- 172.21.15.53:0/2085050446 learned_addr learned my addr 172.21.15.53:0/2085050446 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce27810 0x55bb0ce1e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 -- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce27810 msgr2=0x55bb0ce1e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce27810 0x55bb0ce1e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 -- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 msgr2=0x55bb0ce1e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 -- 172.21.15.53:0/2085050446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80098b0 con 0x55bb0ce25130 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60000675d0 tx=0x7f5ff000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c74da1f08c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5fde7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff002e040 con 0x55bb0ce25130 2023-11-28T11:19:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.000+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb0cd1d140 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ff0003c90 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0020070 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 10 client.5664 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bb0cb7dd90 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb0cd1d140 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5ff0033430 con 0x55bb0ce25130 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 10 client.5664 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 10 client.5664 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 20 client.5664 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0ce47360 0x55bb0ce39580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 4 client.5664 mds_command: new command op to 55007 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb0cbd3f10 con 0x55bb0ce47360 2023-11-28T11:19:51.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.001+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0ce47360 0x55bb0ce39580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.002+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0ce47360 0x55bb0ce39580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add1d7fb20309f0a server_cookie=65cf9ea0ac31ac92 in_seq=0 out_seq=0 2023-11-28T11:19:51.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.002+0000 7f5fde7fc700 10 client.5664 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f5fde7fc700 1 -- 172.21.15.53:0/2085050446 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55bb0cbd3f10 con 0x55bb0ce47360 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f5fde7fc700 10 client.5664 handle_command_reply: tid=0 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 client.5664 shutdown 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 msgr2=0x55bb0ce3c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60000675d0 tx=0x7f5ff000e000 comp rx=0 tx=0).stop 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 shutdown_connections 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0ce47360 0x55bb0ce39580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce3c690 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f5fde7fc700 0 client.5664 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 --2- 172.21.15.53:0/2085050446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce27810 0x55bb0ce1e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 >> 172.21.15.53:0/2085050446 conn(0x55bb0cdfad60 msgr2=0x55bb0cdfd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 shutdown_connections 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 1 -- 172.21.15.53:0/2085050446 wait complete. 2023-11-28T11:19:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.003+0000 7f60192d3b80 20 client.5664 trim_cache size 0 max 16384 2023-11-28T11:19:51.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.021+0000 7f60192d3b80 1 Processor -- start 2023-11-28T11:19:51.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 -- start start 2023-11-28T11:19:51.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce25130 0x55bb0cdf65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cdf7e90 con 0x55bb0ce27810 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0ce1ec60 con 0x55bb0ce28120 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.022+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cdfccc0 con 0x55bb0ce25130 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35760/0 (socket says 172.21.15.53:35760) 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 -- 172.21.15.53:0/1618108477 learned_addr learned my addr 172.21.15.53:0/1618108477 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce25130 0x55bb0cdf65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 -- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce25130 msgr2=0x55bb0cdf65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce25130 0x55bb0cdf65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 -- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 msgr2=0x55bb0ce1d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 -- 172.21.15.53:0/1618108477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb0cd16140 con 0x55bb0ce27810 2023-11-28T11:19:51.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.023+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8018000 tx=0x7f5fe8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de74a1f1fc9fc58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f5ff5ffb700 1 -- 172.21.15.53:0/1618108477 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8023040 con 0x55bb0ce27810 2023-11-28T11:19:51.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f5ff5ffb700 1 -- 172.21.15.53:0/1618108477 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8024040 con 0x55bb0ce27810 2023-11-28T11:19:51.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f5ff5ffb700 1 -- 172.21.15.53:0/1618108477 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8006b10 con 0x55bb0ce27810 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 -- 172.21.15.53:0/1618108477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 msgr2=0x55bb0ce1f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8018000 tx=0x7f5fe8016000 comp rx=0 tx=0).stop 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 -- 172.21.15.53:0/1618108477 shutdown_connections 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce25130 0x55bb0cdf65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1618108477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce27810 0x55bb0ce1f9a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 -- 172.21.15.53:0/1618108477 >> 172.21.15.53:0/1618108477 conn(0x55bb0cdfad60 msgr2=0x55bb0cdf6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 -- 172.21.15.53:0/1618108477 shutdown_connections 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.024+0000 7f60192d3b80 1 -- 172.21.15.53:0/1618108477 wait complete. 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.025+0000 7f60192d3b80 1 -- start start 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.025+0000 7f60192d3b80 1 Processor -- start 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0ce38bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0ce38270 con 0x55bb0ce25130 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0cea8ea0 con 0x55bb0ce27810 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb0ce25620 con 0x55bb0ce28120 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f60192d3b80 10 client.0 _ll_register_callbacks cb 0x55bb0cbf5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35772/0 (socket says 172.21.15.53:35772) 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.026+0000 7f5ffeffd700 1 -- 172.21.15.53:0/1237954701 learned_addr learned my addr 172.21.15.53:0/1237954701 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0ce38bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5fff7fe700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 -- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 msgr2=0x55bb0ce1d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 -- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 msgr2=0x55bb0ce38bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0ce38bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 -- 172.21.15.53:0/1237954701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb0cbd15e0 con 0x55bb0ce25130 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0ce38bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5ffeffd700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00039e0 tx=0x7f5ff00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98970cfee752a6c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5fde7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff002e040 con 0x55bb0ce25130 2023-11-28T11:19:51.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ff0033b80 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.027+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb0cd0d160 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0003e90 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 10 client.5670 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bb0cba4200 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb0cd0d160 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5ff003f430 con 0x55bb0ce25130 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 10 client.5670 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 10 client.5670 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 20 client.5670 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0cea8320 0x55bb0cea86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 4 client.5670 mds_command: new command op to 55007 tid=0[{"prefix": "status"}] 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb0ccf7170 con 0x55bb0cea8320 2023-11-28T11:19:51.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.028+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0cea8320 0x55bb0cea86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f5ff7fff700 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0cea8320 0x55bb0cea86f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6aab5b80881b0c server_cookie=b5846c8bbbd5cbda in_seq=0 out_seq=0 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f5fde7fc700 10 client.5670 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f5fde7fc700 1 -- 172.21.15.53:0/1237954701 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55bb0ccf7170 con 0x55bb0cea8320 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f5fde7fc700 10 client.5670 handle_command_reply: tid=0 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 client.5670 shutdown 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 msgr2=0x55bb0ce38690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00039e0 tx=0x7f5ff00047c0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 shutdown_connections 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55bb0cea8320 0x55bb0cea86f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bb0ce28120 0x55bb0ce1d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb0ce25130 0x55bb0ce38690 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 --2- 172.21.15.53:0/1237954701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb0ce27810 0x55bb0ce38bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 >> 172.21.15.53:0/1237954701 conn(0x55bb0cdfad60 msgr2=0x55bb0ce1f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f5fde7fc700 0 client.5670 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 shutdown_connections 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.029+0000 7f60192d3b80 1 -- 172.21.15.53:0/1237954701 wait complete. 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60192d3b80 20 client.5670 trim_cache size 0 max 16384 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "38901cf1-eb03-4cf6-8c7d-44ad237eb202", 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-28T11:19:51.035 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 55007, 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.625169928, 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 24, 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 48, 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 48, 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.622286288 2023-11-28T11:19:51.036 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 msgr2=0x7f5fe006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 0x7f5fe006a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0008030 comp rx=0 tx=0).stop 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 msgr2=0x7f60001e8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800ca80 tx=0x7f5ff800cd90 comp rx=0 tx=0).stop 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 shutdown_connections 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5fe00682f0 0x7f5fe006a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60000d2320 0x7f6000151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6000161340 0x7f60001e8430 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 --2- 172.21.15.53:0/1715680839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600015c9b0 0x7f6000150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 >> 172.21.15.53:0/1715680839 conn(0x7f6000072600 msgr2=0x7f6000160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 shutdown_connections 2023-11-28T11:19:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.030+0000 7f60059ff700 1 -- 172.21.15.53:0/1715680839 wait complete. 2023-11-28T11:19:51.049 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:19:51.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.250+0000 7efd999ff700 1 Processor -- start 2023-11-28T11:19:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.250+0000 7efd999ff700 1 -- start start 2023-11-28T11:19:51.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.251+0000 7efd999ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 0x7efd940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.251+0000 7efd999ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940d1950 con 0x7efd94161340 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.251+0000 7efd999ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940d24f0 con 0x7efd940cd7c0 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.251+0000 7efd999ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940b5800 con 0x7efd9415c9b0 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd927fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efd94161340 0x7efd940d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34200/0 (socket says 172.21.15.53:34200) 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd927fc700 1 -- 172.21.15.53:0/1035909918 learned_addr learned my addr 172.21.15.53:0/1035909918 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd92ffd700 1 --2- 172.21.15.53:0/1035909918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 0x7efd940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972289883 0 0) 0x7efd940d1950 con 0x7efd94161340 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd7c0035b0 con 0x7efd94161340 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266197634 0 0) 0x7efd940d24f0 con 0x7efd940cd7c0 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd92ffd700 1 -- 172.21.15.53:0/1035909918 >> v1:172.21.15.157:6789/0 conn(0x7efd940cd7c0 legacy=0x7efd940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd92ffd700 1 -- 172.21.15.53:0/1035909918 >> v1:172.21.15.53:6789/0 conn(0x7efd94161340 legacy=0x7efd940d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.252+0000 7efd92ffd700 1 -- 172.21.15.53:0/1035909918 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940b5520 con 0x7efd9415c9b0 2023-11-28T11:19:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.253+0000 7efd92ffd700 1 --2- 172.21.15.53:0/1035909918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 0x7efd940b4bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd84009870 tx=0x7efd84009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b9014c15259f6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.253+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011830 con 0x7efd9415c9b0 2023-11-28T11:19:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.253+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84011990 con 0x7efd9415c9b0 2023-11-28T11:19:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.253+0000 7efd91ffb700 1 -- 172.21.15.53:0/1035909918 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011d00 con 0x7efd9415c9b0 2023-11-28T11:19:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 -- 172.21.15.53:0/1035909918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 msgr2=0x7efd940b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 --2- 172.21.15.53:0/1035909918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 0x7efd940b4bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd84009870 tx=0x7efd84009b80 comp rx=0 tx=0).stop 2023-11-28T11:19:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 -- 172.21.15.53:0/1035909918 shutdown_connections 2023-11-28T11:19:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 --2- 172.21.15.53:0/1035909918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd9415c9b0 0x7efd940b4bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 -- 172.21.15.53:0/1035909918 >> 172.21.15.53:0/1035909918 conn(0x7efd94072600 msgr2=0x7efd940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 -- 172.21.15.53:0/1035909918 shutdown_connections 2023-11-28T11:19:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.254+0000 7efd999ff700 1 -- 172.21.15.53:0/1035909918 wait complete. 2023-11-28T11:19:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.255+0000 7efd999ff700 1 Processor -- start 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.256+0000 7efd999ff700 1 -- start start 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.256+0000 7efd999ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd999ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 0x7efd940c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd999ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 0x7efd941e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd999ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd941e7de0 con 0x7efd940cd7c0 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd999ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd94157a60 con 0x7efd9415c9b0 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd999ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd941e71d0 con 0x7efd94161340 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35782/0 (socket says 172.21.15.53:35782) 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 -- 172.21.15.53:0/51804459 learned_addr learned my addr 172.21.15.53:0/51804459 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd937fe700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 0x7efd941e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd927fc700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 0x7efd940c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 -- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 msgr2=0x7efd941e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 0x7efd941e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 -- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 msgr2=0x7efd940c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 0x7efd940c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 -- 172.21.15.53:0/51804459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840094a0 con 0x7efd940cd7c0 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd937fe700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 0x7efd941e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd927fc700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 0x7efd940c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.257+0000 7efd92ffd700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd84014040 tx=0x7efd84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.258+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840038a0 con 0x7efd940cd7c0 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.258+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8401e070 con 0x7efd940cd7c0 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.258+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84003c60 con 0x7efd940cd7c0 2023-11-28T11:19:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.258+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd940d15e0 con 0x7efd940cd7c0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.258+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd940d1300 con 0x7efd940cd7c0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.259+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd940b5130 con 0x7efd940cd7c0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.260+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efd8400f050 con 0x7efd940cd7c0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.261+0000 7efd7bfff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 0x7efd8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.261+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd84086dc0 con 0x7efd940cd7c0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.261+0000 7efd927fc700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 0x7efd8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.262+0000 7efd927fc700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 0x7efd8006a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd940d2cd0 tx=0x7efd88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.265+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efd84053530 con 0x7efd940cd7c0 2023-11-28T11:19:51.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.512+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd941e80c0 con 0x7efd940cd7c0 2023-11-28T11:19:51.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.512+0000 7efd7bfff700 1 -- 172.21.15.53:0/51804459 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd84015020 con 0x7efd940cd7c0 2023-11-28T11:19:51.514 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 msgr2=0x7efd8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 0x7efd8006a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd940d2cd0 tx=0x7efd88009040 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 msgr2=0x7efd94156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd84014040 tx=0x7efd84004750 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 shutdown_connections 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.514+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efd800682f0 0x7efd8006a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd94161340 0x7efd941e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940cd7c0 0x7efd94156a60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 --2- 172.21.15.53:0/51804459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415c9b0 0x7efd940c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 >> 172.21.15.53:0/51804459 conn(0x7efd94072600 msgr2=0x7efd940b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 shutdown_connections 2023-11-28T11:19:51.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:51.515+0000 7efd999ff700 1 -- 172.21.15.53:0/51804459 wait complete. 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:19:51.535 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:19:51.536 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:51.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.536 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:19:51.580 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42027sec preferred_lft 42027sec 2023-11-28T11:19:51.581 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::892:d6ff:fe9c:c696/64 scope link 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:19:51.582 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::86e:80ff:fe63:4cd1/64 scope link 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:e4:5d:14:b9:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::68e4:5dff:fe14:b995/64 scope link 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:19:51.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.583 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:19:51.687 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:19:51.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.690 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-28T11:19:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-28T11:19:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:19:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:19:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:19:51.736 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:19:51.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.737 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:19:51.738 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:19:51.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:19:51.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:19:51.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:19:51.738 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:19:51.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.912 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:19:51.956 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:19:51.956 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:19:51.957 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:51.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:51.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-11-28T11:19:52.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:19:52.067 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:52.110 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:19:52.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:19:52.111 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:52.194 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:19:52.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:52.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:52.340 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:52.340 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:52.340 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:52.359 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:52.359 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:52.359 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:52.360 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:19:52.360 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:19:52.382 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:19:52.388 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:19:52.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:52.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:19:52.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:52.461 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:19:52.557 INFO:teuthology.orchestra.run.smithi053.stdout:client55112 2023-11-28T11:19:52.558 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:19:52.558 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:19:52.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:52.602 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:19:52.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:19:52.692 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:19:52.737 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:19:52.737 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:19:52.737 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:19:52.738 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:19:52.738 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:19:52.738 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:19:52.892 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2023-11-28T11:19:53.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:19:53.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.316+0000 7fa05e9ff700 1 Processor -- start 2023-11-28T11:19:53.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.316+0000 7fa05e9ff700 1 -- start start 2023-11-28T11:19:53.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa05e9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa058161340 0x7fa0580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa05e9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa0580cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa05e9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0580d1950 con 0x7fa05815c9b0 2023-11-28T11:19:53.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa05e9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0580b5800 con 0x7fa058161340 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa05e9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b5300 con 0x7fa0580d2320 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa057fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa05815c9b0 0x7fa0580b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34208/0 (socket says 172.21.15.53:34208) 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.317+0000 7fa057fff700 1 -- 172.21.15.53:0/535709726 learned_addr learned my addr 172.21.15.53:0/535709726 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa0577fe700 1 --2- 172.21.15.53:0/535709726 >> v2:172.21.15.157:3300/0 conn(0x7fa058161340 0x7fa0580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa0580cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497882052 0 0) 0x7fa0580d1950 con 0x7fa05815c9b0 2023-11-28T11:19:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0400035b0 con 0x7fa05815c9b0 2023-11-28T11:19:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387771626 0 0) 0x7fa0400035b0 con 0x7fa05815c9b0 2023-11-28T11:19:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 msgr2=0x7fa0580cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa0580cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 >> v2:172.21.15.157:3300/0 conn(0x7fa058161340 msgr2=0x7fa0580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 --2- 172.21.15.53:0/535709726 >> v2:172.21.15.157:3300/0 conn(0x7fa058161340 0x7fa0580d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b5520 con 0x7fa05815c9b0 2023-11-28T11:19:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa048001670 con 0x7fa05815c9b0 2023-11-28T11:19:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 >> v1:172.21.15.53:6789/0 conn(0x7fa05815c9b0 legacy=0x7fa0580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa040003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040003fd0 0x7fa040004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa040004d90 0x7fa040005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b5800 con 0x7fa040004d90 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b5300 con 0x7fa040003fd0 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.318+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04000ba90 con 0x7fa0580d2320 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa040004d90 0x7fa040005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa057fff700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa040003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa0577fe700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040003fd0 0x7fa040004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 -- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 msgr2=0x7fa040003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580d2320 0x7fa040003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 -- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040003fd0 msgr2=0x7fa040004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040003fd0 0x7fa040004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 -- 172.21.15.53:0/535709726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400035b0 con 0x7fa040004d90 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.319+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa040004d90 0x7fa040005200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa050009f90 tx=0x7fa040005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d73b2b4981d84c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000ec30 con 0x7fa040004d90 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05000f040 con 0x7fa040004d90 2023-11-28T11:19:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa056ffd700 1 -- 172.21.15.53:0/535709726 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050014480 con 0x7fa040004d90 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 -- 172.21.15.53:0/535709726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa040004d90 msgr2=0x7fa040005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/535709726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa040004d90 0x7fa040005200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa050009f90 tx=0x7fa040005880 comp rx=0 tx=0).stop 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05cf9c700 1 -- 172.21.15.53:0/535709726 reap_dead start 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 -- 172.21.15.53:0/535709726 shutdown_connections 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 -- 172.21.15.53:0/535709726 >> 172.21.15.53:0/535709726 conn(0x7fa058072600 msgr2=0x7fa0580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 -- 172.21.15.53:0/535709726 shutdown_connections 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.320+0000 7fa05e9ff700 1 -- 172.21.15.53:0/535709726 wait complete. 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.322+0000 7fa05e9ff700 1 Processor -- start 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.322+0000 7fa05e9ff700 1 -- start start 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 0x7fa0580c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580c6ca0 0x7fa05814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c7130 con 0x7fa0580cac00 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c7410 con 0x7fa040004d90 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05e9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c76f0 con 0x7fa0580c6ca0 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05cf9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05cf9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35792/0 (socket says 172.21.15.53:35792) 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa05cf9c700 1 -- 172.21.15.53:0/2875945235 learned_addr learned my addr 172.21.15.53:0/2875945235 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.323+0000 7fa0577fe700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580c6ca0 0x7fa05814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa057fff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 0x7fa0580c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 -- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580c6ca0 msgr2=0x7fa05814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580c6ca0 0x7fa05814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 -- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 msgr2=0x7fa0580c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 0x7fa0580c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 -- 172.21.15.53:0/2875945235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580d15e0 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa057fff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 0x7fa0580c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05cf9c700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa050016000 tx=0x7fa050009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050007000 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050021070 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050003e70 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0580d1300 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.324+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05815cd80 con 0x7fa0580cac00 2023-11-28T11:19:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.325+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0580d3f20 con 0x7fa0580cac00 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.326+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa05002d070 con 0x7fa0580cac00 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.327+0000 7fa054ff9700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 0x7fa04406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.327+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa050086f30 con 0x7fa0580cac00 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.327+0000 7fa057fff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 0x7fa04406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.328+0000 7fa057fff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 0x7fa04406a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa048000fb0 tx=0x7fa048009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.332+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0500536a0 con 0x7fa0580cac00 2023-11-28T11:19:53.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.580+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0580745c0 con 0x7fa0580cac00 2023-11-28T11:19:53.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.581+0000 7fa054ff9700 1 -- 172.21.15.53:0/2875945235 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fa050056cc0 con 0x7fa0580cac00 2023-11-28T11:19:53.583 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:53.583 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:19:53.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 msgr2=0x7fa04406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 0x7fa04406a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa048000fb0 tx=0x7fa048009040 comp rx=0 tx=0).stop 2023-11-28T11:19:53.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 msgr2=0x7fa0580cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa050016000 tx=0x7fa050009750 comp rx=0 tx=0).stop 2023-11-28T11:19:53.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 shutdown_connections 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa0440682f0 0x7fa04406a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0580cac00 0x7fa0580cbad0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa040004d90 0x7fa0580c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 --2- 172.21.15.53:0/2875945235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0580c6ca0 0x7fa05814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 >> 172.21.15.53:0/2875945235 conn(0x7fa058072600 msgr2=0x7fa0580b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 shutdown_connections 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.583+0000 7fa05e9ff700 1 -- 172.21.15.53:0/2875945235 wait complete. 2023-11-28T11:19:53.586 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:19:53.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T11:19:53.836 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2023-11-28T11:19:53.837 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:19:53.837 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:19:53.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.921+0000 7f8779ccb080 1 Processor -- start 2023-11-28T11:19:53.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.921+0000 7f8779ccb080 1 -- start start 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92cfddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d084a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92ceeafe0 con 0x55e92cff8920 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92d08a9d0 con 0x55e92d088620 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f8779ccb080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92cfe06f0 con 0x55e92d088250 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35816/0 (socket says 172.21.15.53:35816) 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 -- 172.21.15.53:0/2319208136 learned_addr learned my addr 172.21.15.53:0/2319208136 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 -- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 msgr2=0x55e92cfddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92cfddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 -- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 msgr2=0x55e92d084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d084a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.922+0000 7f876b39b700 1 -- 172.21.15.53:0/2319208136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876400a100 con 0x55e92cff8920 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f876b39b700 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b2e0 tx=0x7f875c00b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222c046ce6fd7336 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8769b98700 1 -- 172.21.15.53:0/2319208136 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c014070 con 0x55e92cff8920 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8769b98700 1 -- 172.21.15.53:0/2319208136 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00f3f0 con 0x55e92cff8920 2023-11-28T11:19:53.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8769b98700 1 -- 172.21.15.53:0/2319208136 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00f760 con 0x55e92cff8920 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 -- 172.21.15.53:0/2319208136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 msgr2=0x55e92d117e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b2e0 tx=0x7f875c00b5f0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 -- 172.21.15.53:0/2319208136 shutdown_connections 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d084a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d117e20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 --2- 172.21.15.53:0/2319208136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92cfddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.923+0000 7f8779ccb080 1 -- 172.21.15.53:0/2319208136 >> 172.21.15.53:0/2319208136 conn(0x55e92cffc9d0 msgr2=0x55e92cfde840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:53.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.924+0000 7f8779ccb080 1 -- 172.21.15.53:0/2319208136 shutdown_connections 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.924+0000 7f8779ccb080 1 -- 172.21.15.53:0/2319208136 wait complete. 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.924+0000 7f8779ccb080 1 Processor -- start 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.924+0000 7f8779ccb080 1 -- start start 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d089dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92d082600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d0845d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92d082b40 con 0x55e92cff8920 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92cfd7c00 con 0x55e92d088620 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.925+0000 7f8779ccb080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92cfdb600 con 0x55e92d088250 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d089dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d089dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35818/0 (socket says 172.21.15.53:35818) 2023-11-28T11:19:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 -- 172.21.15.53:0/304524173 learned_addr learned my addr 172.21.15.53:0/304524173 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876ab9a700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92d082600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876bb9c700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d0845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 -- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 msgr2=0x55e92d082600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e92d088250 0x55e92d082600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 -- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 msgr2=0x55e92d0845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d0845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 -- 172.21.15.53:0/304524173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c001c10 con 0x55e92cff8920 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876bb9c700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e92d088620 0x55e92d0845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f876b39b700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d089dc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b630 tx=0x7f875c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.926+0000 7f8768b96700 1 -- 172.21.15.53:0/304524173 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c014070 con 0x55e92cff8920 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.927+0000 7f8768b96700 1 -- 172.21.15.53:0/304524173 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c001330 con 0x55e92cff8920 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.927+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e92cf03120 con 0x55e92cff8920 2023-11-28T11:19:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.927+0000 7f8768b96700 1 -- 172.21.15.53:0/304524173 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c01b640 con 0x55e92cff8920 2023-11-28T11:19:53.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.927+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e92cef6830 con 0x55e92cff8920 2023-11-28T11:19:53.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.928+0000 7f8768b96700 1 -- 172.21.15.53:0/304524173 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f875c01b7a0 con 0x55e92cff8920 2023-11-28T11:19:53.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.929+0000 7f8768b96700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.929+0000 7f8768b96700 1 -- 172.21.15.53:0/304524173 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f875c00a040 con 0x55e92cff8920 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f8779ccb080 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55e92d082cd0 0x55e92cff5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48) v8 -- 0x55e92cff5780 con 0x55e92d082cd0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f876ab9a700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f876bb9c700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55e92d082cd0 0x55e92cff5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f876ab9a700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8760005fd0 tx=0x7f8760005a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.930+0000 7f876bb9c700 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55e92d082cd0 0x55e92cff5240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f876bb9c700 1 -- 172.21.15.53:0/304524173 <== osd.1 v2:172.21.15.53:6800/132688 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f876400fa60 con 0x55e92d082cd0 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55e92d082cd0 msgr2=0x55e92cff5240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55e92d082cd0 0x55e92cff5240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 msgr2=0x7f87580690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8758066c10 0x7f87580690c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8760005fd0 tx=0x7f8760005a30 comp rx=0 tx=0).stop 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 msgr2=0x55e92d089dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 --2- 172.21.15.53:0/304524173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e92cff8920 0x55e92d089dc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b630 tx=0x7f875c000ed0 comp rx=0 tx=0).stop 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f876bb9c700 1 -- 172.21.15.53:0/304524173 reap_dead start 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 shutdown_connections 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 >> 172.21.15.53:0/304524173 conn(0x55e92cffc9d0 msgr2=0x55e92cfde840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 shutdown_connections 2023-11-28T11:19:53.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:53.931+0000 7f8779ccb080 1 -- 172.21.15.53:0/304524173 wait complete. 2023-11-28T11:19:53.937 INFO:teuthology.orchestra.run.smithi053.stdout:Yalphaparent_a 2023-11-28T11:19:53.937 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "alpha", 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:19:54.118 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "parent_a", 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 8, 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:54.119 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:54.123 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:19:54.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.209+0000 7f4848a67080 1 Processor -- start 2023-11-28T11:19:54.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.209+0000 7f4848a67080 1 -- start start 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a1f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04a057fe0 con 0x55b04a1f5260 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04a164580 con 0x55b04a1f5630 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4848a67080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b04a1edae0 con 0x55b04a165930 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35824/0 (socket says 172.21.15.53:35824) 2023-11-28T11:19:54.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 -- 172.21.15.53:0/1310753698 learned_addr learned my addr 172.21.15.53:0/1310753698 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f483a39b700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f483ab9c700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a1f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 -- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 msgr2=0x55b04a1ed5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1ed5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 -- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 msgr2=0x55b04a1f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a1f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f4839b9a700 1 -- 172.21.15.53:0/1310753698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830005de0 con 0x55b04a1f5260 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f483ab9c700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a1f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.210+0000 7f483a39b700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1ed5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55b04a163fc0 tx=0x7f483000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbba7f5d332235f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4838b98700 1 -- 172.21.15.53:0/1310753698 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830008b60 con 0x55b04a1f5260 2023-11-28T11:19:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4838b98700 1 -- 172.21.15.53:0/1310753698 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830008cc0 con 0x55b04a1f5260 2023-11-28T11:19:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4838b98700 1 -- 172.21.15.53:0/1310753698 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483001c680 con 0x55b04a1f5260 2023-11-28T11:19:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 -- 172.21.15.53:0/1310753698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 msgr2=0x55b04a2851b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55b04a163fc0 tx=0x7f483000d000 comp rx=0 tx=0).stop 2023-11-28T11:19:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 -- 172.21.15.53:0/1310753698 shutdown_connections 2023-11-28T11:19:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1ed5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a1f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 --2- 172.21.15.53:0/1310753698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a2851b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 -- 172.21.15.53:0/1310753698 >> 172.21.15.53:0/1310753698 conn(0x55b04a1699e0 msgr2=0x55b04a14c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.211+0000 7f4848a67080 1 -- 172.21.15.53:0/1310753698 shutdown_connections 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.212+0000 7f4848a67080 1 -- 172.21.15.53:0/1310753698 wait complete. 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.212+0000 7f4848a67080 1 Processor -- start 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.212+0000 7f4848a67080 1 -- start start 2023-11-28T11:19:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a1f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04a14ee90 con 0x55b04a1f5260 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04a1f6bf0 con 0x55b04a1f5630 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4848a67080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b04a1ecc10 con 0x55b04a165930 2023-11-28T11:19:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.213+0000 7f4839b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a1f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a1f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35832/0 (socket says 172.21.15.53:35832) 2023-11-28T11:19:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 -- 172.21.15.53:0/1588210324 learned_addr learned my addr 172.21.15.53:0/1588210324 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f483a39b700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f483ab9c700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 -- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 msgr2=0x55b04a1f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 -- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 msgr2=0x55b04a15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b04a1f5630 0x55b04a15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 -- 172.21.15.53:0/1588210324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b04a08a4f0 con 0x55b04a1f5260 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f483a39b700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b04a165930 0x55b04a1f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f4839b9a700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a1f4130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c00 tx=0x7f4830002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.214+0000 7f482b7fe700 1 -- 172.21.15.53:0/1588210324 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000ad30 con 0x55b04a1f5260 2023-11-28T11:19:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.215+0000 7f482b7fe700 1 -- 172.21.15.53:0/1588210324 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830005580 con 0x55b04a1f5260 2023-11-28T11:19:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.215+0000 7f482b7fe700 1 -- 172.21.15.53:0/1588210324 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483001ce40 con 0x55b04a1f5260 2023-11-28T11:19:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.215+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b04a070120 con 0x55b04a1f5260 2023-11-28T11:19:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.215+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b04a063830 con 0x55b04a1f5260 2023-11-28T11:19:54.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.216+0000 7f482b7fe700 1 -- 172.21.15.53:0/1588210324 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4830012410 con 0x55b04a1f5260 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.217+0000 7f482b7fe700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f481c066c10 0x7f481c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.217+0000 7f482b7fe700 1 -- 172.21.15.53:0/1588210324 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4830029070 con 0x55b04a1f5260 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.217+0000 7f4848a67080 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55b04a1f4670 0x55b04a2983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.217+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x55b04a14e920 con 0x55b04a1f4670 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.218+0000 7f483a39b700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f481c066c10 0x7f481c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.218+0000 7f483ab9c700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55b04a1f4670 0x55b04a2983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.218+0000 7f483ab9c700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55b04a1f4670 0x55b04a2983e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.218+0000 7f483a39b700 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f481c066c10 0x7f481c0690c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e470 tx=0x7f482c003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f483ab9c700 1 -- 172.21.15.53:0/1588210324 <== osd.1 v2:172.21.15.53:6800/132688 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v48'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4834002910 con 0x55b04a1f4670 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55b04a1f4670 msgr2=0x55b04a2983e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55b04a1f4670 0x55b04a2983e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f481c066c10 msgr2=0x7f481c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f481c066c10 0x7f481c0690c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e470 tx=0x7f482c003700 comp rx=0 tx=0).stop 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 msgr2=0x55b04a1f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 --2- 172.21.15.53:0/1588210324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04a1f5260 0x55b04a1f4130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c00 tx=0x7f4830002fb0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f483ab9c700 1 -- 172.21.15.53:0/1588210324 reap_dead start 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 shutdown_connections 2023-11-28T11:19:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 >> 172.21.15.53:0/1588210324 conn(0x55b04a1699e0 msgr2=0x55b04a14c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 shutdown_connections 2023-11-28T11:19:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.220+0000 7f4848a67080 1 -- 172.21.15.53:0/1588210324 wait complete. 2023-11-28T11:19:54.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:19:54.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:19:54.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.442+0000 7f4e03bff700 1 Processor -- start 2023-11-28T11:19:54.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.442+0000 7f4e03bff700 1 -- start start 2023-11-28T11:19:54.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e03bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dfc15a760 0x7f4dfc161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e03bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e03bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc15e0a0 con 0x7f4dfc0cd7c0 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e03bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc15ab30 con 0x7f4dfc15a760 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e03bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc160d70 con 0x7f4dfc15dc30 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e0199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dfc15a760 0x7f4dfc161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e0199b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dfc15a760 0x7f4dfc161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35848/0 (socket says 172.21.15.53:35848) 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.443+0000 7f4e0199b700 1 -- 172.21.15.53:0/4184693406 learned_addr learned my addr 172.21.15.53:0/4184693406 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0119a700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0199b700 1 -- 172.21.15.53:0/4184693406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 msgr2=0x7f4dfc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0199b700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0199b700 1 -- 172.21.15.53:0/4184693406 >> v1:172.21.15.157:6789/0 conn(0x7f4dfc0cd7c0 legacy=0x7f4dfc0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0199b700 1 -- 172.21.15.53:0/4184693406 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0b5130 con 0x7f4dfc15a760 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e0199b700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc161200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098f0 tx=0x7f4df8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833c48e348bb8cc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e00999700 1 -- 172.21.15.53:0/4184693406 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800b8a0 con 0x7f4dfc15a760 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e00999700 1 -- 172.21.15.53:0/4184693406 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800ba00 con 0x7f4dfc15a760 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.444+0000 7f4e00999700 1 -- 172.21.15.53:0/4184693406 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800bd90 con 0x7f4dfc15a760 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 -- 172.21.15.53:0/4184693406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 msgr2=0x7f4dfc161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc161200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098f0 tx=0x7f4df8009c00 comp rx=0 tx=0).stop 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 -- 172.21.15.53:0/4184693406 shutdown_connections 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc161200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 --2- 172.21.15.53:0/4184693406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 -- 172.21.15.53:0/4184693406 >> 172.21.15.53:0/4184693406 conn(0x7f4dfc072620 msgr2=0x7f4dfc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 -- 172.21.15.53:0/4184693406 shutdown_connections 2023-11-28T11:19:54.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.445+0000 7f4e03bff700 1 -- 172.21.15.53:0/4184693406 wait complete. 2023-11-28T11:19:54.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.447+0000 7f4e03bff700 1 Processor -- start 2023-11-28T11:19:54.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.447+0000 7f4e03bff700 1 -- start start 2023-11-28T11:19:54.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dfc0cd7c0 0x7f4dfc1f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b7580 con 0x7f4dfc15a760 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1f5d20 con 0x7f4dfc0cd7c0 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e03bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1f6000 con 0x7f4dfc15dc30 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e0119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e0119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35850/0 (socket says 172.21.15.53:35850) 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e0119a700 1 -- 172.21.15.53:0/2279006347 learned_addr learned my addr 172.21.15.53:0/2279006347 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dfc0cd7c0 0x7f4dfc1f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.448+0000 7f4e0219c700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 -- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 msgr2=0x7f4dfc0b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 -- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dfc0cd7c0 msgr2=0x7f4dfc1f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dfc0cd7c0 0x7f4dfc1f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 -- 172.21.15.53:0/2279006347 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df8009550 con 0x7f4dfc15a760 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e0119a700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d800 tx=0x7f4dec00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4df27fc700 1 -- 172.21.15.53:0/2279006347 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0125f0 con 0x7f4dfc15a760 2023-11-28T11:19:54.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4df27fc700 1 -- 172.21.15.53:0/2279006347 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec012750 con 0x7f4dfc15a760 2023-11-28T11:19:54.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc0b5670 con 0x7f4dfc15a760 2023-11-28T11:19:54.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4df27fc700 1 -- 172.21.15.53:0/2279006347 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec012b00 con 0x7f4dfc15a760 2023-11-28T11:19:54.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.449+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc0b5390 con 0x7f4dfc15a760 2023-11-28T11:19:54.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.452+0000 7f4df27fc700 1 -- 172.21.15.53:0/2279006347 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4dec012c60 con 0x7f4dfc15a760 2023-11-28T11:19:54.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.453+0000 7f4df27fc700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 0x7f4de006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.453+0000 7f4df27fc700 1 -- 172.21.15.53:0/2279006347 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4dec086340 con 0x7f4dfc15a760 2023-11-28T11:19:54.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.454+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 0x7f4de006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.455+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 0x7f4de006a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009c40 tx=0x7f4df8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.455+0000 7f4e174c7b80 1 Processor -- start 2023-11-28T11:19:54.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.455+0000 7f4e174c7b80 1 -- start start 2023-11-28T11:19:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba69fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba6a01f0 0x55b2ba6bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba55fc90 con 0x55b2ba69f8e0 2023-11-28T11:19:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba5ce420 con 0x55b2ba69d200 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba5c7950 con 0x55b2ba6a01f0 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e0199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e0199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35860/0 (socket says 172.21.15.53:35860) 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e0199b700 1 -- 172.21.15.53:0/2742264265 learned_addr learned my addr 172.21.15.53:0/2742264265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e0219c700 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba6a01f0 0x55b2ba6bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.456+0000 7f4e0119a700 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba69fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 -- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba6a01f0 msgr2=0x55b2ba6bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba6a01f0 0x55b2ba6bb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 -- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 msgr2=0x55b2ba69fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba69fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 -- 172.21.15.53:0/2742264265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2ba4496b0 con 0x55b2ba69f8e0 2023-11-28T11:19:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4e0199b700 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b0e0 tx=0x7f4df8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e550742274b2f6d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4ddf7fe700 1 -- 172.21.15.53:0/2742264265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df80038a0 con 0x55b2ba69f8e0 2023-11-28T11:19:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4ddf7fe700 1 -- 172.21.15.53:0/2742264265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8003a00 con 0x55b2ba69f8e0 2023-11-28T11:19:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.457+0000 7f4ddf7fe700 1 -- 172.21.15.53:0/2742264265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8003d70 con 0x55b2ba69f8e0 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 -- 172.21.15.53:0/2742264265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 msgr2=0x55b2ba675af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b0e0 tx=0x7f4df8010000 comp rx=0 tx=0).stop 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 -- 172.21.15.53:0/2742264265 shutdown_connections 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba6a01f0 0x55b2ba6bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba69f8e0 0x55b2ba675af0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/2742264265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba69fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 -- 172.21.15.53:0/2742264265 >> 172.21.15.53:0/2742264265 conn(0x55b2ba672e30 msgr2=0x55b2ba6737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 -- 172.21.15.53:0/2742264265 shutdown_connections 2023-11-28T11:19:54.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.458+0000 7f4e174c7b80 1 -- 172.21.15.53:0/2742264265 wait complete. 2023-11-28T11:19:54.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.459+0000 7f4e174c7b80 1 -- start start 2023-11-28T11:19:54.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.459+0000 7f4e174c7b80 1 Processor -- start 2023-11-28T11:19:54.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba6a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba44b9a0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba6914d0 con 0x55b2ba69f8e0 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba6952a0 con 0x55b2ba69d200 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e174c7b80 10 client.0 _ll_register_callbacks cb 0x55b2ba46d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35874/0 (socket says 172.21.15.53:35874) 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.461+0000 7f4e0219c700 1 -- 172.21.15.53:0/879535911 learned_addr learned my addr 172.21.15.53:0/879535911 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0199b700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba6a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0119a700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 -- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 msgr2=0x55b2ba6a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba6a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 -- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 msgr2=0x55b2ba6b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 -- 172.21.15.53:0/879535911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc0b5130 con 0x55b2ba6a01f0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0199b700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba6a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4e0219c700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011d10 tx=0x7f4df4011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e777ce84b7533f5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4ddcff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4003c10 con 0x55b2ba6a01f0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.462+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4df4003d70 con 0x55b2ba6a01f0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2ba595210 con 0x55b2ba6a01f0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df400f6c0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 10 client.5712 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b2ba3f5e60 con 0x55b2ba6a01f0 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2ba595210 con 0x55b2ba6a01f0 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4df400f820 con 0x55b2ba6a01f0 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 10 client.5712 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 10 client.5712 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 20 client.5712 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba694120 0x55b2ba694520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 4 client.5712 mds_command: new command op to 55007 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.463+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2ba44bfe0 con 0x55b2ba694120 2023-11-28T11:19:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.464+0000 7f4e0199b700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba694120 0x55b2ba694520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.464+0000 7f4e0199b700 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba694120 0x55b2ba694520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651abbb367c88ab1 server_cookie=66085246c0838cc8 in_seq=0 out_seq=0 2023-11-28T11:19:54.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.464+0000 7f4ddcff9700 10 client.5712 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4ddcff9700 1 -- 172.21.15.53:0/879535911 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55b2ba44bfe0 con 0x55b2ba694120 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4ddcff9700 10 client.5712 handle_command_reply: tid=0 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 client.5712 shutdown 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 msgr2=0x55b2ba6b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011d10 tx=0x7f4df4011d40 comp rx=0 tx=0).stop 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 shutdown_connections 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba694120 0x55b2ba694520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba6a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba6b24c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4ddcff9700 0 client.5712 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/879535911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 >> 172.21.15.53:0/879535911 conn(0x55b2ba672e30 msgr2=0x55b2ba674300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 shutdown_connections 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 1 -- 172.21.15.53:0/879535911 wait complete. 2023-11-28T11:19:54.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.465+0000 7f4e174c7b80 20 client.5712 trim_cache size 0 max 16384 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.468+0000 7ff900dff700 1 Processor -- start 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.468+0000 7ff900dff700 1 -- start start 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.469+0000 7ff900dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc15cc80 0x7ff8fc15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.469+0000 7ff900dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.469+0000 7ff900dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0b79c0 con 0x7ff8fc0b6840 2023-11-28T11:19:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.469+0000 7ff900dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0b4610 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.469+0000 7ff900dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0d1570 con 0x7ff8fc15cc80 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35884/0 (socket says 172.21.15.53:35884) 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 -- 172.21.15.53:0/1188663952 learned_addr learned my addr 172.21.15.53:0/1188663952 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fa59c700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc15cc80 0x7ff8fc15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 -- 172.21.15.53:0/1188663952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc15cc80 msgr2=0x7ff8fc15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc15cc80 0x7ff8fc15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 -- 172.21.15.53:0/1188663952 >> v1:172.21.15.157:6789/0 conn(0x7ff8fc0b6840 legacy=0x7ff8fc0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 -- 172.21.15.53:0/1188663952 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0b6db0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.470+0000 7ff8fad9d700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009e50 tx=0x7ff8f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d51f87d4be6ece server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff8f959a700 1 -- 172.21.15.53:0/1188663952 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400e070 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff8f959a700 1 -- 172.21.15.53:0/1188663952 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400b9c0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff8f959a700 1 -- 172.21.15.53:0/1188663952 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400bd70 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 -- 172.21.15.53:0/1188663952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 msgr2=0x7ff8fc0d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009e50 tx=0x7ff8f400b330 comp rx=0 tx=0).stop 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 -- 172.21.15.53:0/1188663952 shutdown_connections 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc0d1030 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 --2- 172.21.15.53:0/1188663952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc15cc80 0x7ff8fc15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 -- 172.21.15.53:0/1188663952 >> 172.21.15.53:0/1188663952 conn(0x7ff8fc09c360 msgr2=0x7ff8fc0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.471+0000 7ff900dff700 1 -- 172.21.15.53:0/1188663952 shutdown_connections 2023-11-28T11:19:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.472+0000 7ff900dff700 1 -- 172.21.15.53:0/1188663952 wait complete. 2023-11-28T11:19:54.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.473+0000 7ff900dff700 1 Processor -- start 2023-11-28T11:19:54.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.473+0000 7ff900dff700 1 -- start start 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc0b6840 0x7ff8fc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8fc15cc80 0x7ff8fc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c8dc0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c81b0 con 0x7ff8fc15cc80 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.474+0000 7ff900dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c8490 con 0x7ff8fc0b6840 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35898/0 (socket says 172.21.15.53:35898) 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 -- 172.21.15.53:0/4077963175 learned_addr learned my addr 172.21.15.53:0/4077963175 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8fa59c700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc0b6840 0x7ff8fc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8fad9d700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8fc15cc80 0x7ff8fc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 -- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc0b6840 msgr2=0x7ff8fc0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8fc0b6840 0x7ff8fc0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 -- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8fc15cc80 msgr2=0x7ff8fc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8fc15cc80 0x7ff8fc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 -- 172.21.15.53:0/4077963175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4009b30 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8fad9d700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8fc15cc80 0x7ff8fc158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8f9d9b700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc0d0bb0 0x7ff8fc1588a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c430 tx=0x7ff8f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.475+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00052d0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.476+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0005450 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.476+0000 7ff900dff700 1 -- 172.21.15.53:0/4077963175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc0b7370 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.476+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0006aa0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.476+0000 7ff900dff700 1 -- 172.21.15.53:0/4077963175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc0b7090 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.476+0000 7ff900dff700 1 -- 172.21.15.53:0/4077963175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff8fc0b7650 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.478+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff8f0022070 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.479+0000 7ff8eaffd700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8dc0682f0 0x7ff8dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.479+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8f0085a40 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.479+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+503 (secure 0 0 0) 0x7ff8f00862b0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.479+0000 7ff900dff700 1 -- 172.21.15.53:0/4077963175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff8fc0678e0 con 0x7ff8fc0d0bb0 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.479+0000 7ff8fa59c700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8dc0682f0 0x7ff8dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.480+0000 7ff8fa59c700 1 --2- 172.21.15.53:0/4077963175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff8dc0682f0 0x7ff8dc06a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec006fd0 tx=0x7ff8ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.487+0000 7f4e174c7b80 1 Processor -- start 2023-11-28T11:19:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.487+0000 7f4e174c7b80 1 -- start start 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba6d3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69f8e0 0x55b2ba68f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba673250 con 0x55b2ba6a01f0 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba6a3bd0 con 0x55b2ba69d200 2023-11-28T11:19:54.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba6b3b20 con 0x55b2ba69f8e0 2023-11-28T11:19:54.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.488+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35902/0 (socket says 172.21.15.53:35902) 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 -- 172.21.15.53:0/1412296374 learned_addr learned my addr 172.21.15.53:0/1412296374 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0119a700 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69f8e0 0x55b2ba68f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0199b700 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba6d3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 -- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69f8e0 msgr2=0x55b2ba68f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69f8e0 0x55b2ba68f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 -- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 msgr2=0x55b2ba6d3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba6d3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 -- 172.21.15.53:0/1412296374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2ba6b4740 con 0x55b2ba6a01f0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000f80 tx=0x7f4df400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd657a382ce30d4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4ddffff700 1 -- 172.21.15.53:0/1412296374 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400f4b0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4ddffff700 1 -- 172.21.15.53:0/1412296374 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df400fd80 con 0x55b2ba6a01f0 2023-11-28T11:19:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.489+0000 7f4ddffff700 1 -- 172.21.15.53:0/1412296374 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400f8c0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1412296374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 msgr2=0x55b2ba68fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000f80 tx=0x7f4df400c000 comp rx=0 tx=0).stop 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1412296374 shutdown_connections 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba68fa70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69d200 0x55b2ba6d3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1412296374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69f8e0 0x55b2ba68f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1412296374 >> 172.21.15.53:0/1412296374 conn(0x55b2ba672e30 msgr2=0x55b2ba6944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1412296374 shutdown_connections 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.490+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1412296374 wait complete. 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.491+0000 7f4e174c7b80 1 -- start start 2023-11-28T11:19:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.491+0000 7f4e174c7b80 1 Processor -- start 2023-11-28T11:19:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba68f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba721140 con 0x55b2ba6a01f0 2023-11-28T11:19:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba720170 con 0x55b2ba69f8e0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2ba6bd3a0 con 0x55b2ba69d200 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.492+0000 7f4e174c7b80 10 client.0 _ll_register_callbacks cb 0x55b2ba46d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35908/0 (socket says 172.21.15.53:35908) 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 -- 172.21.15.53:0/1111686278 learned_addr learned my addr 172.21.15.53:0/1111686278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0199b700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba68f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0119a700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 -- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 msgr2=0x55b2ba68f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba68f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 -- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 msgr2=0x55b2ba6736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 -- 172.21.15.53:0/1111686278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2ba3f5e60 con 0x55b2ba6a01f0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4e0219c700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001220 tx=0x7f4df40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46425b79c3feec49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4ddcff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.493+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400f8c0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4df40050d0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2ba585230 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df4005290 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 10 client.5724 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b2ba3dece0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2ba585230 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4df40053f0 con 0x55b2ba6a01f0 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 10 client.5724 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 10 client.5724 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 20 client.5724 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba673c30 0x55b2ba705ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 4 client.5724 mds_command: new command op to 55007 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:19:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.494+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55b2ba56f240 con 0x55b2ba673c30 2023-11-28T11:19:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.495+0000 7f4e0199b700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba673c30 0x55b2ba705ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.495+0000 7f4e0199b700 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba673c30 0x55b2ba705ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96bff2d7ccf6ac0 server_cookie=539e734340ca3862 in_seq=0 out_seq=0 2023-11-28T11:19:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.495+0000 7f4ddcff9700 10 client.5724 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4ddcff9700 1 -- 172.21.15.53:0/1111686278 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b2ba56f240 con 0x55b2ba673c30 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4ddcff9700 10 client.5724 handle_command_reply: tid=0 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 client.5724 shutdown 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 msgr2=0x55b2ba705980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001220 tx=0x7f4df40014a0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 shutdown_connections 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b2ba673c30 0x55b2ba705ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2ba69d200 0x55b2ba68f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2ba6a01f0 0x55b2ba705980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 --2- 172.21.15.53:0/1111686278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2ba69f8e0 0x55b2ba6736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 >> 172.21.15.53:0/1111686278 conn(0x55b2ba672e30 msgr2=0x55b2ba6a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4ddcff9700 0 client.5724 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.504 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f76479ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={55112=0-4194304@1} caps={55112=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=55112 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5604cf9b0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:19:54.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 shutdown_connections 2023-11-28T11:19:54.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.496+0000 7f4e174c7b80 1 -- 172.21.15.53:0/1111686278 wait complete. 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e174c7b80 20 client.5724 trim_cache size 0 max 16384 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "a51df350-6954-4b53-a86d-ae9f4a79904b", 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2023-11-28T11:19:54.505 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 msgr2=0x7f4de006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 0x7f4de006a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009c40 tx=0x7f4df8008030 comp rx=0 tx=0).stop 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 msgr2=0x7f4dfc1577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d800 tx=0x7f4dec00dbc0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 shutdown_connections 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4de00682f0 0x7f4de006a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dfc15dc30 0x7f4dfc0b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dfc0cd7c0 0x7f4dfc1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 --2- 172.21.15.53:0/2279006347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dfc15a760 0x7f4dfc1577e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 >> 172.21.15.53:0/2279006347 conn(0x7f4dfc072620 msgr2=0x7f4dfc158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 shutdown_connections 2023-11-28T11:19:54.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.497+0000 7f4e03bff700 1 -- 172.21.15.53:0/2279006347 wait complete. 2023-11-28T11:19:54.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:19:54.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.722+0000 7fc5727ff700 1 Processor -- start 2023-11-28T11:19:54.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.722+0000 7fc5727ff700 1 -- start start 2023-11-28T11:19:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc5727ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc5727ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0d1950 con 0x7fc56c15c9b0 2023-11-28T11:19:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc5727ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0ce140 con 0x7fc56c161340 2023-11-28T11:19:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc5727ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0b5800 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc570d9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc570d9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35912/0 (socket says 172.21.15.53:35912) 2023-11-28T11:19:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc56bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc56c15c9b0 0x7fc56c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34222/0 (socket says 172.21.15.53:34222) 2023-11-28T11:19:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.723+0000 7fc570d9c700 1 -- 172.21.15.53:0/3882241000 learned_addr learned my addr 172.21.15.53:0/3882241000 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc570d9c700 1 -- 172.21.15.53:0/3882241000 >> v1:172.21.15.157:6789/0 conn(0x7fc56c161340 legacy=0x7fc56c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc570d9c700 1 -- 172.21.15.53:0/3882241000 >> v1:172.21.15.53:6789/0 conn(0x7fc56c15c9b0 legacy=0x7fc56c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc570d9c700 1 -- 172.21.15.53:0/3882241000 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0d24f0 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc570d9c700 1 --2- 172.21.15.53:0/3882241000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc564009e50 tx=0x7fc56400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fede5908f3d92f2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc56affd700 1 -- 172.21.15.53:0/3882241000 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400e070 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc56affd700 1 -- 172.21.15.53:0/3882241000 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400b9c0 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.724+0000 7fc56affd700 1 -- 172.21.15.53:0/3882241000 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400bd70 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 -- 172.21.15.53:0/3882241000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c0cd7c0 msgr2=0x7fc56c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3882241000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc564009e50 tx=0x7fc56400b330 comp rx=0 tx=0).stop 2023-11-28T11:19:54.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 -- 172.21.15.53:0/3882241000 shutdown_connections 2023-11-28T11:19:54.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3882241000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c0cd7c0 0x7fc56c0cdc00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 -- 172.21.15.53:0/3882241000 >> 172.21.15.53:0/3882241000 conn(0x7fc56c072600 msgr2=0x7fc56c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 -- 172.21.15.53:0/3882241000 shutdown_connections 2023-11-28T11:19:54.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.725+0000 7fc5727ff700 1 -- 172.21.15.53:0/3882241000 wait complete. 2023-11-28T11:19:54.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.727+0000 7fc5727ff700 1 Processor -- start 2023-11-28T11:19:54.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.728+0000 7fc5727ff700 1 -- start start 2023-11-28T11:19:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc56c0cd7c0 0x7fc56c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c161340 0x7fc56c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0cd170 con 0x7fc56c15c9b0 2023-11-28T11:19:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1e66f0 con 0x7fc56c161340 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc5727ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1e69d0 con 0x7fc56c0cd7c0 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc56b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc56b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35922/0 (socket says 172.21.15.53:35922) 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc56b7fe700 1 -- 172.21.15.53:0/3385516834 learned_addr learned my addr 172.21.15.53:0/3385516834 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc56c0cd7c0 0x7fc56c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.729+0000 7fc570d9c700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c161340 0x7fc56c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 -- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc56c0cd7c0 msgr2=0x7fc56c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc56c0cd7c0 0x7fc56c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 -- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c161340 msgr2=0x7fc56c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c161340 0x7fc56c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 -- 172.21.15.53:0/3385516834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564009b30 con 0x7fc56c15c9b0 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc56000c4e0 tx=0x7fc56000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc568ff9700 1 -- 172.21.15.53:0/3385516834 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600052d0 con 0x7fc56c15c9b0 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc568ff9700 1 -- 172.21.15.53:0/3385516834 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc560005430 con 0x7fc56c15c9b0 2023-11-28T11:19:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc568ff9700 1 -- 172.21.15.53:0/3385516834 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600038c0 con 0x7fc56c15c9b0 2023-11-28T11:19:54.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c0d1300 con 0x7fc56c15c9b0 2023-11-28T11:19:54.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.730+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c0b5520 con 0x7fc56c15c9b0 2023-11-28T11:19:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.732+0000 7fc568ff9700 1 -- 172.21.15.53:0/3385516834 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc560019030 con 0x7fc56c15c9b0 2023-11-28T11:19:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.733+0000 7fc568ff9700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 0x7fc55806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.733+0000 7fc568ff9700 1 -- 172.21.15.53:0/3385516834 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc560084fe0 con 0x7fc56c15c9b0 2023-11-28T11:19:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.734+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 0x7fc55806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.734+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 0x7fc55806a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000f60 tx=0x7fc55c007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.735+0000 7fc5861ddb80 1 Processor -- start 2023-11-28T11:19:54.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.735+0000 7fc5861ddb80 1 -- start start 2023-11-28T11:19:54.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40ee75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee76170 0x55b40ee917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40ed35c10 con 0x55b40ee75860 2023-11-28T11:19:54.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40eda43a0 con 0x55b40ee73180 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40ed9d8d0 con 0x55b40ee76170 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc56bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc56bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35930/0 (socket says 172.21.15.53:35930) 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.736+0000 7fc56bfff700 1 -- 172.21.15.53:0/2248636804 learned_addr learned my addr 172.21.15.53:0/2248636804 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc570d9c700 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee76170 0x55b40ee917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40ee75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 -- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee76170 msgr2=0x55b40ee917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee76170 0x55b40ee917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 -- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 msgr2=0x55b40ee75c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40ee75c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 -- 172.21.15.53:0/2248636804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40ec1f630 con 0x55b40ee75860 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc56bfff700 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0015c0 tx=0x7fc55c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad114fa1e2e5833 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc5527fc700 1 -- 172.21.15.53:0/2248636804 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c001d40 con 0x55b40ee75860 2023-11-28T11:19:54.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc5527fc700 1 -- 172.21.15.53:0/2248636804 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c0133f0 con 0x55b40ee75860 2023-11-28T11:19:54.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.737+0000 7fc5527fc700 1 -- 172.21.15.53:0/2248636804 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c013760 con 0x55b40ee75860 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 -- 172.21.15.53:0/2248636804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 msgr2=0x55b40ee4ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0015c0 tx=0x7fc55c013000 comp rx=0 tx=0).stop 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 -- 172.21.15.53:0/2248636804 shutdown_connections 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee76170 0x55b40ee917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ba70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/2248636804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40ee75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 -- 172.21.15.53:0/2248636804 >> 172.21.15.53:0/2248636804 conn(0x55b40ee48db0 msgr2=0x55b40ee49740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 -- 172.21.15.53:0/2248636804 shutdown_connections 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.738+0000 7fc5861ddb80 1 -- 172.21.15.53:0/2248636804 wait complete. 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.739+0000 7fc5861ddb80 1 -- start start 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.739+0000 7fc5861ddb80 1 Processor -- start 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40eea39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee75860 0x55b40eea3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40ec21920 con 0x55b40ee76170 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40ee65f00 con 0x55b40ee73180 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40ee74920 con 0x55b40ee75860 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.740+0000 7fc5861ddb80 10 client.0 _ll_register_callbacks cb 0x55b40ec43290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35936/0 (socket says 172.21.15.53:35936) 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 -- 172.21.15.53:0/3262486837 learned_addr learned my addr 172.21.15.53:0/3262486837 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40eea39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee75860 0x55b40eea3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 -- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee75860 msgr2=0x55b40eea3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee75860 0x55b40eea3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 -- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 msgr2=0x55b40eea39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40eea39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 -- 172.21.15.53:0/3262486837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c0d1950 con 0x55b40ee76170 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc570d9c700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc564012000 tx=0x7fc564004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db701453c8d04ddc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc54b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564004240 con 0x55b40ee76170 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5640043a0 con 0x55b40ee76170 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.741+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b40ed6b190 con 0x55b40ee76170 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5640045a0 con 0x55b40ee76170 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 10 client.5739 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b40ebcbde0 con 0x55b40ee76170 2023-11-28T11:19:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b40ed6b190 con 0x55b40ee76170 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc56400b050 con 0x55b40ee76170 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 10 client.5739 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 10 client.5739 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 20 client.5739 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee64770 0x55b40ee73650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 4 client.5739 mds_command: new command op to 55007 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:19:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b40ec21f60 con 0x55b40ee64770 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.742+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee64770 0x55b40ee73650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.743+0000 7fc56bfff700 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee64770 0x55b40ee73650 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558e70b31ed98b19 server_cookie=8c9fbbcce3488c0c in_seq=0 out_seq=0 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.743+0000 7fc54b7fe700 10 client.5739 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.743+0000 7fc54b7fe700 1 -- 172.21.15.53:0/3262486837 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55b40ec21f60 con 0x55b40ee64770 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.743+0000 7fc54b7fe700 10 client.5739 handle_command_reply: tid=0 2023-11-28T11:19:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 client.5739 shutdown 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 msgr2=0x55b40ee63f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc564012000 tx=0x7fc564004b40 comp rx=0 tx=0).stop 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 shutdown_connections 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee64770 0x55b40ee73650 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee63f20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee73180 0x55b40eea39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/3262486837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee75860 0x55b40eea3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc54b7fe700 0 client.5739 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 >> 172.21.15.53:0/3262486837 conn(0x55b40ee48db0 msgr2=0x55b40ee4b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 shutdown_connections 2023-11-28T11:19:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 1 -- 172.21.15.53:0/3262486837 wait complete. 2023-11-28T11:19:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.744+0000 7fc5861ddb80 20 client.5739 trim_cache size 0 max 16384 2023-11-28T11:19:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.762+0000 7fc5861ddb80 1 Processor -- start 2023-11-28T11:19:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.762+0000 7fc5861ddb80 1 -- start start 2023-11-28T11:19:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.762+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40ee47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee75860 0x55b40ee67740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40ee745f0 con 0x55b40ee76170 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40eea3fa0 con 0x55b40ee75860 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40ee46de0 con 0x55b40ee73180 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35946/0 (socket says 172.21.15.53:35946) 2023-11-28T11:19:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 -- 172.21.15.53:0/632133329 learned_addr learned my addr 172.21.15.53:0/632133329 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc56bfff700 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40ee47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee75860 0x55b40ee67740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 -- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 msgr2=0x55b40ee47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40ee47bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 -- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee75860 msgr2=0x55b40ee67740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee75860 0x55b40ee67740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 -- 172.21.15.53:0/632133329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40ee67530 con 0x55b40ee76170 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.763+0000 7fc570d9c700 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc564002230 tx=0x7fc56401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f44e13bcd167 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc552ffd700 1 -- 172.21.15.53:0/632133329 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564004890 con 0x55b40ee76170 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc552ffd700 1 -- 172.21.15.53:0/632133329 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400b0c0 con 0x55b40ee76170 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc552ffd700 1 -- 172.21.15.53:0/632133329 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564004240 con 0x55b40ee76170 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 -- 172.21.15.53:0/632133329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 msgr2=0x55b40ee67c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc564002230 tx=0x7fc56401f040 comp rx=0 tx=0).stop 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 -- 172.21.15.53:0/632133329 shutdown_connections 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40ee47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee76170 0x55b40ee67c80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/632133329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee75860 0x55b40ee67740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 -- 172.21.15.53:0/632133329 >> 172.21.15.53:0/632133329 conn(0x55b40ee48db0 msgr2=0x55b40ee63cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 -- 172.21.15.53:0/632133329 shutdown_connections 2023-11-28T11:19:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.764+0000 7fc5861ddb80 1 -- 172.21.15.53:0/632133329 wait complete. 2023-11-28T11:19:54.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.765+0000 7fc5861ddb80 1 -- start start 2023-11-28T11:19:54.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.765+0000 7fc5861ddb80 1 Processor -- start 2023-11-28T11:19:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40eef5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee76170 0x55b40ee4b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40eef7cd0 con 0x55b40ee75860 2023-11-28T11:19:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40eea3dd0 con 0x55b40ee76170 2023-11-28T11:19:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40ee47c70 con 0x55b40ee73180 2023-11-28T11:19:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc5861ddb80 10 client.0 _ll_register_callbacks cb 0x55b40ec43290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:19:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc56b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc56b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35950/0 (socket says 172.21.15.53:35950) 2023-11-28T11:19:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.767+0000 7fc56b7fe700 1 -- 172.21.15.53:0/4190250274 learned_addr learned my addr 172.21.15.53:0/4190250274 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:19:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc570d9c700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee76170 0x55b40ee4b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56bfff700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40eef5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 -- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 msgr2=0x55b40eef5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40eef5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 -- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee76170 msgr2=0x55b40ee4b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:19:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee76170 0x55b40ee4b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 -- 172.21.15.53:0/4190250274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b40ebcbde0 con 0x55b40ee75860 2023-11-28T11:19:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc56b7fe700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc560055110 tx=0x7fc56005d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347dab0b7adb0cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:19:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc54b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:19:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560053870 con 0x55b40ee75860 2023-11-28T11:19:54.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc560068b50 con 0x55b40ee75860 2023-11-28T11:19:54.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.768+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b40ed5b1b0 con 0x55b40ee75860 2023-11-28T11:19:54.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc560083250 con 0x55b40ee75860 2023-11-28T11:19:54.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 10 client.5745 fetch_fsmap learned FSMap version 24 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b40ebb4c60 con 0x55b40ee75860 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b40ed5b1b0 con 0x55b40ee75860 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc560068d10 con 0x55b40ee75860 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 10 client.5745 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 10 client.5745 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 20 client.5745 populate_metadata read hostname 'smithi053' 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee6d0b0 0x55b40ee444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 4 client.5745 mds_command: new command op to 55007 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b40ed451c0 con 0x55b40ee6d0b0 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.769+0000 7fc56bfff700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee6d0b0 0x55b40ee444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc56bfff700 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee6d0b0 0x55b40ee444a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94dc05131e3494ba server_cookie=e1e360d6264b39b3 in_seq=0 out_seq=0 2023-11-28T11:19:54.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc54b7fe700 10 client.5745 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc54b7fe700 1 -- 172.21.15.53:0/4190250274 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55b40ed451c0 con 0x55b40ee6d0b0 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc54b7fe700 10 client.5745 handle_command_reply: tid=0 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 client.5745 shutdown 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 msgr2=0x55b40ee4ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc560055110 tx=0x7fc56005d620 comp rx=0 tx=0).stop 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 shutdown_connections 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x55b40ee6d0b0 0x55b40ee444a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b40ee73180 0x55b40eef5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b40ee76170 0x55b40ee4b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc54b7fe700 0 client.5745 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:19:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 --2- 172.21.15.53:0/4190250274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40ee75860 0x55b40ee4ada0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 >> 172.21.15.53:0/4190250274 conn(0x55b40ee48db0 msgr2=0x55b40ee67740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 shutdown_connections 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 1 -- 172.21.15.53:0/4190250274 wait complete. 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.770+0000 7fc5861ddb80 20 client.5745 trim_cache size 0 max 16384 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:19:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "a51df350-6954-4b53-a86d-ae9f4a79904b": { 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "a51df350-6954-4b53-a86d-ae9f4a79904b", 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:19:54.788 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 msgr2=0x7fc55806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 0x7fc55806a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000f60 tx=0x7fc55c007ed0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 msgr2=0x7fc56c0cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc56000c4e0 tx=0x7fc56000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 shutdown_connections 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5580682f0 0x7fc55806a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc56c0cd7c0 0x7fc56c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c161340 0x7fc56c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 --2- 172.21.15.53:0/3385516834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc56c15c9b0 0x7fc56c0cc6f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:19:54.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 >> 172.21.15.53:0/3385516834 conn(0x7fc56c072600 msgr2=0x7fc56c0b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:19:54.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 shutdown_connections 2023-11-28T11:19:54.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:54.771+0000 7fc5727ff700 1 -- 172.21.15.53:0/3385516834 wait complete. 2023-11-28T11:19:54.791 INFO:tasks.cephfs.filesystem:scrub status for tag:a51df350-6954-4b53-a86d-ae9f4a79904b - {'path': '/', 'tag': 'a51df350-6954-4b53-a86d-ae9f4a79904b', 'options': 'recursive,repair'} 2023-11-28T11:19:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:55.827+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T11:19:54.497387+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7ff8f0085020 con 0x7ff8fc0d0bb0 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: cluster: 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: id: 38901cf1-eb03-4cf6-8c7d-44ad237eb202 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: health: HEALTH_OK 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: services: 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: mgr: x(active, since 2m), standbys: y 2023-11-28T11:19:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: data: 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: volumes: 1/1 healthy 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: objects: 24 objects, 579 KiB 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: usage: 104 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: pgs: 105 active+clean 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: io: 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: client: 1.6 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T11:19:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.497387+0000 mds.c [INF] scrub queued for path: / 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.497391+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.497395+0000 mds.c [INF] scrub summary: active paths [/] 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.498364+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.498377+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:19:55.832 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:54.498901+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:19:59.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:19:59.839+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T11:19:58.414099+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff8f0086290 con 0x7ff8fc0d0bb0 2023-11-28T11:19:59.841 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:19:58.414099+0000 mds.c [INF] scrub summary: idle 2023-11-28T11:20:00.843 INFO:teuthology.orchestra.run.smithi053.stdout:2023-11-28T11:20:00.000187+0000 mon.a [INF] overall HEALTH_OK 2023-11-28T11:20:00.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:00.841+0000 7ff8eaffd700 1 -- 172.21.15.53:0/4077963175 <== mon.0 v2:172.21.15.53:3300/0 9 ==== log(1 entries from seq 433 at 2023-11-28T11:20:00.000187+0000) v1 ==== 187+0+0 (secure 0 0 0) 0x7ff8f005df00 con 0x7ff8fc0d0bb0 2023-11-28T11:20:14.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:14.915 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:24.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:20:25.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.009+0000 7f42495ff700 1 Processor -- start 2023-11-28T11:20:25.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.009+0000 7f42495ff700 1 -- start start 2023-11-28T11:20:25.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.010+0000 7f42495ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f424415c9b0 0x7f42440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.010+0000 7f42495ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4244161340 0x7f42440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.010+0000 7f42495ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440d1950 con 0x7f42440d2320 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.010+0000 7f42495ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440b5800 con 0x7f4244161340 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.010+0000 7f42495ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440b5520 con 0x7f424415c9b0 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42437fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f42440d2320 0x7f42440cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34044/0 (socket says 172.21.15.53:34044) 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42437fe700 1 -- 172.21.15.53:0/3786595568 learned_addr learned my addr 172.21.15.53:0/3786595568 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 --2- 172.21.15.53:0/3786595568 >> v2:172.21.15.53:3300/0 conn(0x7f4244161340 0x7f42440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f4242ffd700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f424415c9b0 0x7f42440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f4241ffb700 1 -- 172.21.15.53:0/3786595568 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426121860 0 0) 0x7f42440d1950 con 0x7f42440d2320 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 -- 172.21.15.53:0/3786595568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f424415c9b0 msgr2=0x7f42440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f424415c9b0 0x7f42440b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 -- 172.21.15.53:0/3786595568 >> v1:172.21.15.53:6789/0 conn(0x7f42440d2320 legacy=0x7f42440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 -- 172.21.15.53:0/3786595568 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440d1300 con 0x7f4244161340 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.011+0000 7f42427fc700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244161340 0x7f42440d1de0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4238009840 tx=0x7f4238009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9b6a1898398e4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f4241ffb700 1 -- 172.21.15.53:0/3786595568 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011830 con 0x7f4244161340 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f4241ffb700 1 -- 172.21.15.53:0/3786595568 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238011990 con 0x7f4244161340 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f4241ffb700 1 -- 172.21.15.53:0/3786595568 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011d20 con 0x7f4244161340 2023-11-28T11:20:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 -- 172.21.15.53:0/3786595568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244161340 msgr2=0x7f42440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244161340 0x7f42440d1de0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4238009840 tx=0x7f4238009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 -- 172.21.15.53:0/3786595568 shutdown_connections 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f424415c9b0 0x7f42440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 --2- 172.21.15.53:0/3786595568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244161340 0x7f42440d1de0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 -- 172.21.15.53:0/3786595568 >> 172.21.15.53:0/3786595568 conn(0x7f4244072600 msgr2=0x7f42440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 -- 172.21.15.53:0/3786595568 shutdown_connections 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.012+0000 7f42495ff700 1 -- 172.21.15.53:0/3786595568 wait complete. 2023-11-28T11:20:25.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.014+0000 7f42495ff700 1 Processor -- start 2023-11-28T11:20:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.014+0000 7f42495ff700 1 -- start start 2023-11-28T11:20:25.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42440d2320 0x7f424415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4244161340 0x7f4244156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441570a0 con 0x7f424415c9b0 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441e6460 con 0x7f42440d2320 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.015+0000 7f42495ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42441e6740 con 0x7f4244161340 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44288/0 (socket says 172.21.15.53:44288) 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 -- 172.21.15.53:0/1586009719 learned_addr learned my addr 172.21.15.53:0/1586009719 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42437fe700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4244161340 0x7f4244156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42440d2320 0x7f424415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 -- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4244161340 msgr2=0x7f4244156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4244161340 0x7f4244156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 -- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42440d2320 msgr2=0x7f424415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42440d2320 0x7f424415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 -- 172.21.15.53:0/1586009719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42380094a0 con 0x7f424415c9b0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.016+0000 7f42427fc700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f4238004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.017+0000 7f422b7fe700 1 -- 172.21.15.53:0/1586009719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380038a0 con 0x7f424415c9b0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.017+0000 7f422b7fe700 1 -- 172.21.15.53:0/1586009719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423801f070 con 0x7f424415c9b0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.017+0000 7f422b7fe700 1 -- 172.21.15.53:0/1586009719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238003c40 con 0x7f424415c9b0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.017+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42440b5130 con 0x7f424415c9b0 2023-11-28T11:20:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.017+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42440d15e0 con 0x7f424415c9b0 2023-11-28T11:20:25.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.020+0000 7f422b7fe700 1 -- 172.21.15.53:0/1586009719 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f423800f050 con 0x7f424415c9b0 2023-11-28T11:20:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.020+0000 7f422b7fe700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 0x7f423006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.021+0000 7f422b7fe700 1 -- 172.21.15.53:0/1586009719 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4238021070 con 0x7f424415c9b0 2023-11-28T11:20:25.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.021+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 0x7f423006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.022+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 0x7f423006a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4234009b70 tx=0x7f4234008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.022+0000 7f425ce42b80 1 Processor -- start 2023-11-28T11:20:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.022+0000 7f425ce42b80 1 -- start start 2023-11-28T11:20:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133c8f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133b33c10 con 0x556133c71180 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133ba23a0 con 0x556133c74170 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.023+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133b9b8d0 con 0x556133c73860 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44294/0 (socket says 172.21.15.53:44294) 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 -- 172.21.15.53:0/267999665 learned_addr learned my addr 172.21.15.53:0/267999665 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42437fe700 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133c8f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f4242ffd700 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 -- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 msgr2=0x556133c49a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c49a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 -- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 msgr2=0x556133c8f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133c8f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 -- 172.21.15.53:0/267999665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556133a1d630 con 0x556133c71180 2023-11-28T11:20:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f42427fc700 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4238085d00 tx=0x7f4238053ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d7972e24b08f0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f4228ff9700 1 -- 172.21.15.53:0/267999665 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238010380 con 0x556133c71180 2023-11-28T11:20:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f4228ff9700 1 -- 172.21.15.53:0/267999665 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423805fbf0 con 0x556133c71180 2023-11-28T11:20:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.024+0000 7f4228ff9700 1 -- 172.21.15.53:0/267999665 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423805f760 con 0x556133c71180 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 -- 172.21.15.53:0/267999665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 msgr2=0x556133c73c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4238085d00 tx=0x7f4238053ea0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 -- 172.21.15.53:0/267999665 shutdown_connections 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c49a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133c8f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 --2- 172.21.15.53:0/267999665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c73c30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 -- 172.21.15.53:0/267999665 >> 172.21.15.53:0/267999665 conn(0x556133c46db0 msgr2=0x556133c47740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 -- 172.21.15.53:0/267999665 shutdown_connections 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.025+0000 7f425ce42b80 1 -- 172.21.15.53:0/267999665 wait complete. 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.026+0000 7f425ce42b80 1 -- start start 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.026+0000 7f425ce42b80 1 Processor -- start 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c71180 0x556133c86620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c6b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133a1f920 con 0x556133c74170 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c93d50 con 0x556133c71180 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133c86f20 con 0x556133c73860 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f425ce42b80 10 client.0 _ll_register_callbacks cb 0x556133a41290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f42437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f42437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44308/0 (socket says 172.21.15.53:44308) 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f42437fe700 1 -- 172.21.15.53:0/1523157168 learned_addr learned my addr 172.21.15.53:0/1523157168 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c71180 0x556133c86620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.027+0000 7f42427fc700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c6b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 -- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 msgr2=0x556133c6b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c6b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 -- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c71180 msgr2=0x556133c86620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c71180 0x556133c86620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 -- 172.21.15.53:0/1523157168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42440d15e0 con 0x556133c74170 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42427fc700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c6b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f42437fe700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f423c007630 tx=0x7f423c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0ac9db3e6c7d1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f4221ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c010ae0 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f423c010c40 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.028+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556133b69190 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f423c010e00 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 10 client.5760 fetch_fsmap learned FSMap version 24 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5561339c9de0 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556133b69190 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f423c014400 con 0x556133c74170 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 10 client.5760 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 10 client.5760 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 20 client.5760 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:25.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133c6a5a0 0x556133c92d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 4 client.5760 mds_command: new command op to 55007 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:20:25.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556133a1ff60 con 0x556133c6a5a0 2023-11-28T11:20:25.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.029+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133c6a5a0 0x556133c92d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.030+0000 7f4242ffd700 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133c6a5a0 0x556133c92d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60e158a351a8f69 server_cookie=5010204d653257a0 in_seq=0 out_seq=0 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.030+0000 7f4221ffb700 10 client.5760 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.030+0000 7f4221ffb700 1 -- 172.21.15.53:0/1523157168 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x556133a1ff60 con 0x556133c6a5a0 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.030+0000 7f4221ffb700 10 client.5760 handle_command_reply: tid=0 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.030+0000 7f425ce42b80 1 client.5760 shutdown 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 msgr2=0x556133c6b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f423c007630 tx=0x7f423c007940 comp rx=0 tx=0).stop 2023-11-28T11:20:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 shutdown_connections 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133c6a5a0 0x556133c92d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c74170 0x556133c6b980 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c71180 0x556133c86620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 --2- 172.21.15.53:0/1523157168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c6b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 >> 172.21.15.53:0/1523157168 conn(0x556133c46db0 msgr2=0x556133c42f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f4221ffb700 0 client.5760 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 shutdown_connections 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 1 -- 172.21.15.53:0/1523157168 wait complete. 2023-11-28T11:20:25.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.031+0000 7f425ce42b80 20 client.5760 trim_cache size 0 max 16384 2023-11-28T11:20:25.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.048+0000 7f425ce42b80 1 Processor -- start 2023-11-28T11:20:25.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.048+0000 7f425ce42b80 1 -- start start 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133cee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c697a0 con 0x556133c71180 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133ca11f0 con 0x556133c74170 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133c6b860 con 0x556133c73860 2023-11-28T11:20:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44314/0 (socket says 172.21.15.53:44314) 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f4242ffd700 1 -- 172.21.15.53:0/737230923 learned_addr learned my addr 172.21.15.53:0/737230923 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42437fe700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133cee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 -- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 msgr2=0x556133c929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.049+0000 7f42427fc700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 -- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 msgr2=0x556133cee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133cee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 -- 172.21.15.53:0/737230923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556133b62190 con 0x556133c71180 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42437fe700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133cee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:25.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42427fc700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:25.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f4242ffd700 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4234011f60 tx=0x7f4234014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c0935c9f15546a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42297fa700 1 -- 172.21.15.53:0/737230923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423402adc0 con 0x556133c71180 2023-11-28T11:20:25.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42297fa700 1 -- 172.21.15.53:0/737230923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423402a330 con 0x556133c71180 2023-11-28T11:20:25.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.050+0000 7f42297fa700 1 -- 172.21.15.53:0/737230923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234015640 con 0x556133c71180 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 -- 172.21.15.53:0/737230923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 msgr2=0x556133c47b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4234011f60 tx=0x7f4234014000 comp rx=0 tx=0).stop 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 -- 172.21.15.53:0/737230923 shutdown_connections 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c74170 0x556133cee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133c47b00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 --2- 172.21.15.53:0/737230923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c73860 0x556133c929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 -- 172.21.15.53:0/737230923 >> 172.21.15.53:0/737230923 conn(0x556133c46db0 msgr2=0x556133c91b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 -- 172.21.15.53:0/737230923 shutdown_connections 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.051+0000 7f425ce42b80 1 -- 172.21.15.53:0/737230923 wait complete. 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.052+0000 7f425ce42b80 1 -- start start 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.052+0000 7f425ce42b80 1 Processor -- start 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c73860 0x556133c93ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 0x556133c94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133c49870 con 0x556133c71180 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556133ceb250 con 0x556133c73860 2023-11-28T11:20:25.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x556133c493e0 con 0x556133c74170 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f425ce42b80 10 client.0 _ll_register_callbacks cb 0x556133a41290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44324/0 (socket says 172.21.15.53:44324) 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 -- 172.21.15.53:0/3279384109 learned_addr learned my addr 172.21.15.53:0/3279384109 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f42427fc700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c73860 0x556133c93ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f42437fe700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 0x556133c94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 -- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 msgr2=0x556133c94420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 0x556133c94420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 -- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c73860 msgr2=0x556133c93ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c73860 0x556133c93ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f4242ffd700 1 -- 172.21.15.53:0/3279384109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4234011c90 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.053+0000 7f42437fe700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 0x556133c94420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f4242ffd700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4234009a30 tx=0x7f4234004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377bc8ae61b0a95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f4221ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234015710 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4234028070 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556133b591b0 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4234015870 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f425ce42b80 10 client.5766 fetch_fsmap learned FSMap version 24 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.054+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5561339f0250 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556133b591b0 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4234032030 con 0x556133c71180 2023-11-28T11:20:25.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 10 client.5766 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 10 client.5766 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 20 client.5766 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133ceaa70 0x556133c87060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 4 client.5766 mds_command: new command op to 55007 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 --> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556133b431c0 con 0x556133ceaa70 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.055+0000 7f42427fc700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133ceaa70 0x556133c87060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f42427fc700 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133ceaa70 0x556133c87060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fd7076b9d2deac server_cookie=18b448ca655d0507 in_seq=0 out_seq=0 2023-11-28T11:20:25.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f4221ffb700 10 client.5766 ms_handle_connect on v2:172.21.15.53:6833/4065027310 2023-11-28T11:20:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f4221ffb700 1 -- 172.21.15.53:0/3279384109 <== mds.0 v2:172.21.15.53:6833/4065027310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556133b431c0 con 0x556133ceaa70 2023-11-28T11:20:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f4221ffb700 10 client.5766 handle_command_reply: tid=0 2023-11-28T11:20:25.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 client.5766 shutdown 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 msgr2=0x556133cea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4234009a30 tx=0x7f4234004750 comp rx=0 tx=0).stop 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 shutdown_connections 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:6833/4065027310,v1:172.21.15.53:6835/4065027310] conn(0x556133ceaa70 0x556133c87060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556133c74170 0x556133c94420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556133c71180 0x556133cea3e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f4221ffb700 0 client.5766 ms_handle_reset on v2:172.21.15.53:6833/4065027310 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 --2- 172.21.15.53:0/3279384109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556133c73860 0x556133c93ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.056+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 >> 172.21.15.53:0/3279384109 conn(0x556133c46db0 msgr2=0x556133c929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 shutdown_connections 2023-11-28T11:20:25.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f425ce42b80 1 -- 172.21.15.53:0/3279384109 wait complete. 2023-11-28T11:20:25.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f425ce42b80 20 client.5766 trim_cache size 0 max 16384 2023-11-28T11:20:25.065 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:25.065 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2023-11-28T11:20:25.065 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2023-11-28T11:20:25.065 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 msgr2=0x7f423006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 0x7f423006a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4234009b70 tx=0x7f4234008040 comp rx=0 tx=0).stop 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 msgr2=0x7f4244156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f4238004750 comp rx=0 tx=0).stop 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 shutdown_connections 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f42300682f0 0x7f423006a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4244161340 0x7f4244156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42440d2320 0x7f424415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 --2- 172.21.15.53:0/1586009719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f424415c9b0 0x7f4244156620 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 >> 172.21.15.53:0/1586009719 conn(0x7f4244072600 msgr2=0x7f42440b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 shutdown_connections 2023-11-28T11:20:25.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.057+0000 7f42495ff700 1 -- 172.21.15.53:0/1586009719 wait complete. 2023-11-28T11:20:25.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:20:25.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:25.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.377+0000 7fc6619ff700 1 Processor -- start 2023-11-28T11:20:25.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.378+0000 7fc6619ff700 1 -- start start 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc6619ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc65c06c7a0 0x7fc65c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc6619ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c0b1090 con 0x7fc65c0b0c90 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc6619ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c15bc00 con 0x7fc65c158480 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc6619ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c0cb0b0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc653fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc65c06c7a0 0x7fc65c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc65b7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc65c0b0c90 0x7fc65c070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34050/0 (socket says 172.21.15.53:34050) 2023-11-28T11:20:25.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.379+0000 7fc65b7fe700 1 -- 172.21.15.53:0/4103655405 learned_addr learned my addr 172.21.15.53:0/4103655405 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc653fff700 1 -- 172.21.15.53:0/4103655405 >> v1:172.21.15.157:6789/0 conn(0x7fc65c158480 legacy=0x7fc65c0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc653fff700 1 -- 172.21.15.53:0/4103655405 >> v1:172.21.15.53:6789/0 conn(0x7fc65c0b0c90 legacy=0x7fc65c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc653fff700 1 -- 172.21.15.53:0/4103655405 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c06b9e0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc653fff700 1 --2- 172.21.15.53:0/4103655405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c0b0750 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc644009840 tx=0x7fc644009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418ef39f19fe12ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc65a7fc700 1 -- 172.21.15.53:0/4103655405 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011830 con 0x7fc65c06c7a0 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc65a7fc700 1 -- 172.21.15.53:0/4103655405 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644011990 con 0x7fc65c06c7a0 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.380+0000 7fc65a7fc700 1 -- 172.21.15.53:0/4103655405 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011d20 con 0x7fc65c06c7a0 2023-11-28T11:20:25.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 -- 172.21.15.53:0/4103655405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 msgr2=0x7fc65c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 --2- 172.21.15.53:0/4103655405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c0b0750 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc644009840 tx=0x7fc644009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 -- 172.21.15.53:0/4103655405 shutdown_connections 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 --2- 172.21.15.53:0/4103655405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c0b0750 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 -- 172.21.15.53:0/4103655405 >> 172.21.15.53:0/4103655405 conn(0x7fc65c0cb750 msgr2=0x7fc65c0cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 -- 172.21.15.53:0/4103655405 shutdown_connections 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.381+0000 7fc6619ff700 1 -- 172.21.15.53:0/4103655405 wait complete. 2023-11-28T11:20:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.383+0000 7fc6619ff700 1 Processor -- start 2023-11-28T11:20:25.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.383+0000 7fc6619ff700 1 -- start start 2023-11-28T11:20:25.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65c0b0c90 0x7fc65c0c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65c158480 0x7fc65c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0ad7b0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0ac8c0 con 0x7fc65c0b0c90 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc6619ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0acba0 con 0x7fc65c158480 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44346/0 (socket says 172.21.15.53:44346) 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 -- 172.21.15.53:0/821905162 learned_addr learned my addr 172.21.15.53:0/821905162 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc653fff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65c0b0c90 0x7fc65c0c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65b7fe700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65c158480 0x7fc65c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 -- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65c158480 msgr2=0x7fc65c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65c158480 0x7fc65c0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 -- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65c0b0c90 msgr2=0x7fc65c0c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65c0b0c90 0x7fc65c0c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.384+0000 7fc65affd700 1 -- 172.21.15.53:0/821905162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440094a0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc65affd700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c450 tx=0x7fc64c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0052d0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c005430 con 0x7fc65c06c7a0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0038c0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c06bfa0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.385+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c06bcc0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.386+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c06c280 con 0x7fc65c06c7a0 2023-11-28T11:20:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.387+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc64c019030 con 0x7fc65c06c7a0 2023-11-28T11:20:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.388+0000 7fc658ff9700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 0x7fc63c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.388+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc64c085000 con 0x7fc65c06c7a0 2023-11-28T11:20:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.388+0000 7fc653fff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 0x7fc63c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.389+0000 7fc653fff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 0x7fc63c06a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc65c06cb90 tx=0x7fc644008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.392+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc64c051770 con 0x7fc65c06c7a0 2023-11-28T11:20:25.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.636+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc65c0745c0 con 0x7fc65c06c7a0 2023-11-28T11:20:25.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.637+0000 7fc658ff9700 1 -- 172.21.15.53:0/821905162 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fc64c017090 con 0x7fc65c06c7a0 2023-11-28T11:20:25.638 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:25.638 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:20:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 msgr2=0x7fc63c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 0x7fc63c06a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc65c06cb90 tx=0x7fc644008030 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 msgr2=0x7fc65c156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c450 tx=0x7fc64c00c760 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 shutdown_connections 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc63c0682f0 0x7fc63c06a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65c158480 0x7fc65c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc65c06c7a0 0x7fc65c156d80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 --2- 172.21.15.53:0/821905162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65c0b0c90 0x7fc65c0c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 >> 172.21.15.53:0/821905162 conn(0x7fc65c0cb750 msgr2=0x7fc65c0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.639+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 shutdown_connections 2023-11-28T11:20:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.640+0000 7fc6619ff700 1 -- 172.21.15.53:0/821905162 wait complete. 2023-11-28T11:20:25.642 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:20:25.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T11:20:25.889 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2023-11-28T11:20:25.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:20:25.890 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:20:25.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.975+0000 7f8a4a999080 1 Processor -- start 2023-11-28T11:20:25.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.975+0000 7f8a4a999080 1 -- start start 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c688fdcfe0 con 0x55c68917a250 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68917b5a0 con 0x55c68917a620 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a4a999080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6890cf6f0 con 0x55c6890ea920 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44360/0 (socket says 172.21.15.53:44360) 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1870317087 learned_addr learned my addr 172.21.15.53:0/1870317087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3c79c700 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3bf9b700 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 msgr2=0x55c689175150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 msgr2=0x55c6890cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1870317087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a30005de0 con 0x55c68917a250 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.976+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a30015040 tx=0x7f8a3000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ae6ec5b51bba69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a3a798700 1 -- 172.21.15.53:0/1870317087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30008b60 con 0x55c68917a250 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a3a798700 1 -- 172.21.15.53:0/1870317087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a30008cc0 con 0x55c68917a250 2023-11-28T11:20:25.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a3a798700 1 -- 172.21.15.53:0/1870317087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30012650 con 0x55c68917a250 2023-11-28T11:20:25.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 -- 172.21.15.53:0/1870317087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 msgr2=0x55c6890cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a30015040 tx=0x7f8a3000d000 comp rx=0 tx=0).stop 2023-11-28T11:20:25.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 -- 172.21.15.53:0/1870317087 shutdown_connections 2023-11-28T11:20:25.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1870317087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c6890cf9f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 -- 172.21.15.53:0/1870317087 >> 172.21.15.53:0/1870317087 conn(0x55c6890ee9d0 msgr2=0x55c6890d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 -- 172.21.15.53:0/1870317087 shutdown_connections 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.977+0000 7f8a4a999080 1 -- 172.21.15.53:0/1870317087 wait complete. 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.978+0000 7f8a4a999080 1 Processor -- start 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.978+0000 7f8a4a999080 1 -- start start 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c68916cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6891730e0 con 0x55c68917a250 2023-11-28T11:20:25.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6890e8b90 con 0x55c68917a620 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a4a999080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c68916d700 con 0x55c6890ea920 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a3b79a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c68916cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a3b79a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c68916cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44374/0 (socket says 172.21.15.53:44374) 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.979+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1231491197 learned_addr learned my addr 172.21.15.53:0/1231491197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3bf9b700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3c79c700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 msgr2=0x55c689175150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c6890ea920 0x55c689175150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 msgr2=0x55c6890cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c68917a620 0x55c6890cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 -- 172.21.15.53:0/1231491197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c68900f4f0 con 0x55c68917a250 2023-11-28T11:20:25.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a3b79a700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c68916cda0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000a5a0 tx=0x7f8a30002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a39796700 1 -- 172.21.15.53:0/1231491197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30005580 con 0x55c68917a250 2023-11-28T11:20:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a39796700 1 -- 172.21.15.53:0/1231491197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a30012bb0 con 0x55c68917a250 2023-11-28T11:20:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a39796700 1 -- 172.21.15.53:0/1231491197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3002d440 con 0x55c68917a250 2023-11-28T11:20:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.980+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c688ff5120 con 0x55c68917a250 2023-11-28T11:20:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.981+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c688fe8830 con 0x55c68917a250 2023-11-28T11:20:25.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.982+0000 7f8a39796700 1 -- 172.21.15.53:0/1231491197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a30022070 con 0x55c68917a250 2023-11-28T11:20:25.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.983+0000 7f8a39796700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a28066c10 0x7f8a280690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.983+0000 7f8a39796700 1 -- 172.21.15.53:0/1231491197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a300875a0 con 0x55c68917a250 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.983+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55c6890d0130 0x55c68916fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.983+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48) v8 -- 0x55c689170380 con 0x55c6890d0130 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.983+0000 7f8a3bf9b700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a28066c10 0x7f8a280690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.984+0000 7f8a3c79c700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55c6890d0130 0x55c68916fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.984+0000 7f8a3c79c700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55c6890d0130 0x55c68916fe40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.984+0000 7f8a3bf9b700 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a28066c10 0x7f8a280690c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c689170d40 tx=0x7f8a2c003070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a3c79c700 1 -- 172.21.15.53:0/1231491197 <== osd.1 v2:172.21.15.53:6800/132688 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f8a34002960 con 0x55c6890d0130 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55c6890d0130 msgr2=0x55c68916fe40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x55c6890d0130 0x55c68916fe40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a28066c10 msgr2=0x7f8a280690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8a28066c10 0x7f8a280690c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c689170d40 tx=0x7f8a2c003070 comp rx=0 tx=0).stop 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 msgr2=0x55c68916cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 --2- 172.21.15.53:0/1231491197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c68917a250 0x55c68916cda0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000a5a0 tx=0x7f8a30002fb0 comp rx=0 tx=0).stop 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a3c79c700 1 -- 172.21.15.53:0/1231491197 reap_dead start 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 shutdown_connections 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 >> 172.21.15.53:0/1231491197 conn(0x55c6890ee9d0 msgr2=0x55c6890d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 shutdown_connections 2023-11-28T11:20:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:25.985+0000 7f8a4a999080 1 -- 172.21.15.53:0/1231491197 wait complete. 2023-11-28T11:20:25.990 INFO:teuthology.orchestra.run.smithi053.stdout:Yalphaparent_a 2023-11-28T11:20:25.990 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2023-11-28T11:20:26.101 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "alpha", 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "parent_a", 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 8, 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:26.102 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:26.108 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:20:26.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:20:26.196 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:20:26.290 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:20:26.290 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:26.290 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:26.335 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:26.335 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:26.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:26.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.661+0000 7fb7415ff700 1 Processor -- start 2023-11-28T11:20:26.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.661+0000 7fb7415ff700 1 -- start start 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb7415ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb73c159490 0x7fb73c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb7415ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb73c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb7415ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c0ccc00 con 0x7fb73c159860 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb7415ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c0ccec0 con 0x7fb73c159490 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb7415ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0cd1a0 con 0x7fb73c0cdc80 2023-11-28T11:20:26.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb73a7fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb73c159860 0x7fb73c0c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34062/0 (socket says 172.21.15.53:34062) 2023-11-28T11:20:26.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.662+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3709452817 learned_addr learned my addr 172.21.15.53:0/3709452817 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:26.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb73b7fe700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb73c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb73affd700 1 --2- 172.21.15.53:0/3709452817 >> v2:172.21.15.157:3300/0 conn(0x7fb73c159490 0x7fb73c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606838255 0 0) 0x7fb73c0ccc00 con 0x7fb73c159860 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7240035b0 con 0x7fb73c159860 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540705432 0 0) 0x7fb7240035b0 con 0x7fb73c159860 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 msgr2=0x7fb73c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb73c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 >> v2:172.21.15.157:3300/0 conn(0x7fb73c159490 msgr2=0x7fb73c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 --2- 172.21.15.53:0/3709452817 >> v2:172.21.15.157:3300/0 conn(0x7fb73c159490 0x7fb73c0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c0cd480 con 0x7fb73c159860 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fb730001670 con 0x7fb73c159860 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 >> v1:172.21.15.53:6789/0 conn(0x7fb73c159860 legacy=0x7fb73c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb724003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb724003f80 0x7fb724004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0ccec0 con 0x7fb724003f80 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0cd1a0 con 0x7fb724004d90 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.663+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72400dc50 con 0x7fb73c0cdc80 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb724003f80 0x7fb724004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73affd700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb724003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73b7fe700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 msgr2=0x7fb724003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c0cdc80 0x7fb724003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724004d90 msgr2=0x7fb724005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3709452817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240035b0 con 0x7fb724003f80 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.664+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb724003f80 0x7fb724004850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c370 tx=0x7fb724005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74f00f98a59518d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730007c60 con 0x7fb724003f80 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000ce50 con 0x7fb724003f80 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb739ffb700 1 -- 172.21.15.53:0/3709452817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730005ba0 con 0x7fb724003f80 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 -- 172.21.15.53:0/3709452817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb724003f80 msgr2=0x7fb724004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3709452817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb724003f80 0x7fb724004850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c370 tx=0x7fb724005880 comp rx=0 tx=0).stop 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb73b7fe700 1 -- 172.21.15.53:0/3709452817 reap_dead start 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 -- 172.21.15.53:0/3709452817 shutdown_connections 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 -- 172.21.15.53:0/3709452817 >> 172.21.15.53:0/3709452817 conn(0x7fb73c0cb670 msgr2=0x7fb73c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 -- 172.21.15.53:0/3709452817 shutdown_connections 2023-11-28T11:20:26.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.665+0000 7fb7415ff700 1 -- 172.21.15.53:0/3709452817 wait complete. 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.667+0000 7fb7415ff700 1 Processor -- start 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.667+0000 7fb7415ff700 1 -- start start 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724003f80 0x7fb73c1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 0x7fb73c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c15b9a0 con 0x7fb73c15b140 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c15c7c0 con 0x7fb724003f80 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb7415ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c15bbb0 con 0x7fb73c15b5b0 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.668+0000 7fb73a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44392/0 (socket says 172.21.15.53:44392) 2023-11-28T11:20:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3505488630 learned_addr learned my addr 172.21.15.53:0/3505488630 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73affd700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724003f80 0x7fb73c1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73b7fe700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 0x7fb73c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 msgr2=0x7fb73c1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 0x7fb73c1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724003f80 msgr2=0x7fb73c1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724003f80 0x7fb73c1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 -- 172.21.15.53:0/3505488630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c0b0bb0 con 0x7fb73c15b140 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73b7fe700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 0x7fb73c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb73a7fc700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb730011000 tx=0x7fb730000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300070d0 con 0x7fb73c15b140 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730007230 con 0x7fb73c15b140 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300075e0 con 0x7fb73c15b140 2023-11-28T11:20:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.669+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c0cd760 con 0x7fb73c15b140 2023-11-28T11:20:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.670+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c0b0d40 con 0x7fb73c15b140 2023-11-28T11:20:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.671+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c15c950 con 0x7fb73c15b140 2023-11-28T11:20:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.671+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb730024030 con 0x7fb73c15b140 2023-11-28T11:20:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.673+0000 7fb7237fe700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 0x7fb72806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.673+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb730087260 con 0x7fb73c15b140 2023-11-28T11:20:26.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.673+0000 7fb73affd700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 0x7fb72806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:26.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.674+0000 7fb73affd700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 0x7fb72806a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009c60 tx=0x7fb72c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:26.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.677+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7300539d0 con 0x7fb73c15b140 2023-11-28T11:20:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.883+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb73c0745c0 con 0x7fb73c15b140 2023-11-28T11:20:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.884+0000 7fb7237fe700 1 -- 172.21.15.53:0/3505488630 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17979 (secure 0 0 0) 0x7fb730007dc0 con 0x7fb73c15b140 2023-11-28T11:20:26.885 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:26.885 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:45.403641+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 msgr2=0x7fb72806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 0x7fb72806a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009c60 tx=0x7fb72c009210 comp rx=0 tx=0).stop 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 msgr2=0x7fb73c157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb730011000 tx=0x7fb730000fe0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 shutdown_connections 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb7280682f0 0x7fb72806a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73c15b5b0 0x7fb73c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb724003f80 0x7fb73c1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 --2- 172.21.15.53:0/3505488630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73c15b140 0x7fb73c157260 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 >> 172.21.15.53:0/3505488630 conn(0x7fb73c0cb670 msgr2=0x7fb73c070910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 shutdown_connections 2023-11-28T11:20:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:26.887+0000 7fb7415ff700 1 -- 172.21.15.53:0/3505488630 wait complete. 2023-11-28T11:20:26.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:19:42.218381+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:19:43.389004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:20:26.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:20:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.110+0000 7f35641ff700 1 Processor -- start 2023-11-28T11:20:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.110+0000 7f35641ff700 1 -- start start 2023-11-28T11:20:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f35641ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f355c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f35641ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c0d1950 con 0x7f355c161340 2023-11-28T11:20:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f35641ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c0ce140 con 0x7f355c15c9b0 2023-11-28T11:20:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f35641ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b5800 con 0x7f355c0cd7c0 2023-11-28T11:20:27.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f356179a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f355c161340 0x7f355c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34076/0 (socket says 172.21.15.53:34076) 2023-11-28T11:20:27.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.111+0000 7f356179a700 1 -- 172.21.15.53:0/3741820548 learned_addr learned my addr 172.21.15.53:0/3741820548 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f355c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555899315 0 0) 0x7f355c0d1950 con 0x7f355c161340 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35480035b0 con 0x7f355c161340 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909464041 0 0) 0x7f355c0ce140 con 0x7f355c15c9b0 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f355c0d1950 con 0x7f355c15c9b0 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884866841 0 0) 0x7f35480035b0 con 0x7f355c161340 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 msgr2=0x7f355c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f355c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 >> v1:172.21.15.157:6789/0 conn(0x7f355c15c9b0 legacy=0x7f355c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0d24f0 con 0x7f355c161340 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f354c001670 con 0x7f355c161340 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f355c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 >> v1:172.21.15.53:6789/0 conn(0x7f355c161340 legacy=0x7f355c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.112+0000 7f3560f99700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548003bc0 0x7f3548003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3560f99700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f3548004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3560f99700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3548004d90 0x7f3548005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b5800 con 0x7f3548004d90 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354800bb40 con 0x7f3548003bc0 2023-11-28T11:20:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354800bd00 con 0x7f355c0cd7c0 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3548004d90 0x7f3548005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356179a700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f3548004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3561f9b700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548003bc0 0x7f3548003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 -- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 msgr2=0x7f3548004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f3548004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 -- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548003bc0 msgr2=0x7f3548003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548003bc0 0x7f3548003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 -- 172.21.15.53:0/3741820548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554009a70 con 0x7f3548004d90 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356179a700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c0cd7c0 0x7f3548004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f3561f9b700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548003bc0 0x7f3548003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.113+0000 7f356279c700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3548004d90 0x7f3548005200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f355400e350 tx=0x7f3548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2afbeb9556976b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400edf0 con 0x7f3548004d90 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400f040 con 0x7f3548004d90 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f3560f99700 1 -- 172.21.15.53:0/3741820548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35540146a0 con 0x7f3548004d90 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 -- 172.21.15.53:0/3741820548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3548004d90 msgr2=0x7f3548005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 --2- 172.21.15.53:0/3741820548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3548004d90 0x7f3548005200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f355400e350 tx=0x7f3548005880 comp rx=0 tx=0).stop 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f356279c700 1 -- 172.21.15.53:0/3741820548 reap_dead start 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 -- 172.21.15.53:0/3741820548 shutdown_connections 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 -- 172.21.15.53:0/3741820548 >> 172.21.15.53:0/3741820548 conn(0x7f355c072600 msgr2=0x7f355c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 -- 172.21.15.53:0/3741820548 shutdown_connections 2023-11-28T11:20:27.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.114+0000 7f35641ff700 1 -- 172.21.15.53:0/3741820548 wait complete. 2023-11-28T11:20:27.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.116+0000 7f35641ff700 1 Processor -- start 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.116+0000 7f35641ff700 1 -- start start 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 0x7f355c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c157910 0x7f355c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0caaa0 con 0x7f355c0cb900 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cc580 con 0x7f3548004d90 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.117+0000 7f35641ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cc2a0 con 0x7f355c157910 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44416/0 (socket says 172.21.15.53:44416) 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 -- 172.21.15.53:0/1551971865 learned_addr learned my addr 172.21.15.53:0/1551971865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356179a700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c157910 0x7f355c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f3561f9b700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 0x7f355c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 -- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c157910 msgr2=0x7f355c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c157910 0x7f355c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 -- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 msgr2=0x7f355c0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 0x7f355c0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 -- 172.21.15.53:0/1551971865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0d1300 con 0x7f355c0cb900 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f3561f9b700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 0x7f355c0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f356279c700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3554016000 tx=0x7f3554009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554009410 con 0x7f355c0cb900 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3554004470 con 0x7f355c0cb900 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.118+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c0b5520 con 0x7f355c0cb900 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.119+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554004860 con 0x7f355c0cb900 2023-11-28T11:20:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.119+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0d15e0 con 0x7f355c0cb900 2023-11-28T11:20:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.120+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0ca220 con 0x7f355c0cb900 2023-11-28T11:20:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.121+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3554014480 con 0x7f355c0cb900 2023-11-28T11:20:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.122+0000 7f35527fc700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 0x7f354006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.122+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3554087f40 con 0x7f355c0cb900 2023-11-28T11:20:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.122+0000 7f3561f9b700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 0x7f354006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.123+0000 7f3561f9b700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 0x7f354006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3558000f60 tx=0x7f3558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.126+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3554054650 con 0x7f355c0cb900 2023-11-28T11:20:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.327+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f355c0745c0 con 0x7f355c0cb900 2023-11-28T11:20:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.327+0000 7f35527fc700 1 -- 172.21.15.53:0/1551971865 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3554018020 con 0x7f355c0cb900 2023-11-28T11:20:27.328 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 msgr2=0x7f354006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 0x7f354006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3558000f60 tx=0x7f3558009040 comp rx=0 tx=0).stop 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 msgr2=0x7f355c15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3554016000 tx=0x7f3554009750 comp rx=0 tx=0).stop 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 shutdown_connections 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f35400682f0 0x7f354006a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f355c0cb900 0x7f355c15e7c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3548004d90 0x7f355c0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 --2- 172.21.15.53:0/1551971865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355c157910 0x7f355c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 >> 172.21.15.53:0/1551971865 conn(0x7f355c072600 msgr2=0x7f355c15a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 shutdown_connections 2023-11-28T11:20:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.330+0000 7f35641ff700 1 -- 172.21.15.53:0/1551971865 wait complete. 2023-11-28T11:20:27.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:20:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.553+0000 7f50c23ff700 1 Processor -- start 2023-11-28T11:20:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.553+0000 7f50c23ff700 1 -- start start 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.554+0000 7f50c23ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50bc161340 0x7f50bc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.554+0000 7f50c23ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 0x7f50bc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c23ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0d1950 con 0x7f50bc15c9b0 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c23ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0b5800 con 0x7f50bc161340 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c23ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0b5300 con 0x7f50bc0d2320 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50bbfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f50bc15c9b0 0x7f50bc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34080/0 (socket says 172.21.15.53:34080) 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50bbfff700 1 -- 172.21.15.53:0/1328299904 learned_addr learned my addr 172.21.15.53:0/1328299904 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c099c700 1 --2- 172.21.15.53:0/1328299904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 0x7f50bc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50bb7fe700 1 --2- 172.21.15.53:0/1328299904 >> v2:172.21.15.157:3300/0 conn(0x7f50bc161340 0x7f50bc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50baffd700 1 -- 172.21.15.53:0/1328299904 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967337531 0 0) 0x7f50bc0d1950 con 0x7f50bc15c9b0 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50baffd700 1 -- 172.21.15.53:0/1328299904 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a40035b0 con 0x7f50bc15c9b0 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c099c700 1 -- 172.21.15.53:0/1328299904 >> v2:172.21.15.157:3300/0 conn(0x7f50bc161340 msgr2=0x7f50bc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c099c700 1 --2- 172.21.15.53:0/1328299904 >> v2:172.21.15.157:3300/0 conn(0x7f50bc161340 0x7f50bc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.555+0000 7f50c099c700 1 -- 172.21.15.53:0/1328299904 >> v1:172.21.15.53:6789/0 conn(0x7f50bc15c9b0 legacy=0x7f50bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.556+0000 7f50c099c700 1 -- 172.21.15.53:0/1328299904 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0b5520 con 0x7f50bc0d2320 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.556+0000 7f50c099c700 1 --2- 172.21.15.53:0/1328299904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 0x7f50bc0cd7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23395eb08a3ee9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.556+0000 7f50baffd700 1 -- 172.21.15.53:0/1328299904 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400e070 con 0x7f50bc0d2320 2023-11-28T11:20:27.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.556+0000 7f50baffd700 1 -- 172.21.15.53:0/1328299904 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400bb20 con 0x7f50bc0d2320 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.556+0000 7f50baffd700 1 -- 172.21.15.53:0/1328299904 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400be90 con 0x7f50bc0d2320 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 -- 172.21.15.53:0/1328299904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 msgr2=0x7f50bc0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 --2- 172.21.15.53:0/1328299904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 0x7f50bc0cd7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 -- 172.21.15.53:0/1328299904 shutdown_connections 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 --2- 172.21.15.53:0/1328299904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc0d2320 0x7f50bc0cd7c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 --2- 172.21.15.53:0/1328299904 >> v2:172.21.15.157:3300/0 conn(0x7f50bc161340 0x7f50bc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 -- 172.21.15.53:0/1328299904 >> 172.21.15.53:0/1328299904 conn(0x7f50bc072600 msgr2=0x7f50bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 -- 172.21.15.53:0/1328299904 shutdown_connections 2023-11-28T11:20:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.557+0000 7f50c23ff700 1 -- 172.21.15.53:0/1328299904 wait complete. 2023-11-28T11:20:27.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.559+0000 7f50c23ff700 1 Processor -- start 2023-11-28T11:20:27.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.559+0000 7f50c23ff700 1 -- start start 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 0x7f50bc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc161340 0x7f50bc1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc151b40 con 0x7f50bc0d2320 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc156540 con 0x7f50bc15c9b0 2023-11-28T11:20:27.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c23ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc154750 con 0x7f50bc161340 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44428/0 (socket says 172.21.15.53:44428) 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50bbfff700 1 -- 172.21.15.53:0/3531138397 learned_addr learned my addr 172.21.15.53:0/3531138397 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50bb7fe700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 0x7f50bc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.560+0000 7f50c099c700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc161340 0x7f50bc1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 -- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc161340 msgr2=0x7f50bc1f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc161340 0x7f50bc1f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 -- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 msgr2=0x7f50bc143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 0x7f50bc143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 -- 172.21.15.53:0/3531138397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b4009a70 con 0x7f50bc0d2320 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bb7fe700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 0x7f50bc143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50bbfff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c450 tx=0x7f50ac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac005d70 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac0070d0 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac007440 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc0d15e0 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.561+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc0d1300 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.562+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc144270 con 0x7f50bc0d2320 2023-11-28T11:20:27.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.563+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f50ac019030 con 0x7f50bc0d2320 2023-11-28T11:20:27.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.565+0000 7f50b8ff9700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 0x7f50a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.565+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50ac084f80 con 0x7f50bc0d2320 2023-11-28T11:20:27.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.565+0000 7f50bb7fe700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 0x7f50a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:27.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.566+0000 7f50bb7fe700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 0x7f50a806a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50bc15fa70 tx=0x7f50b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:27.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.569+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50ac04b5e0 con 0x7f50bc0d2320 2023-11-28T11:20:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.769+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f50bc15cd80 con 0x7f50bc0d2320 2023-11-28T11:20:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.770+0000 7f50b8ff9700 1 -- 172.21.15.53:0/3531138397 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f50ac017020 con 0x7f50bc0d2320 2023-11-28T11:20:27.771 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:20:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 msgr2=0x7f50a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 0x7f50a806a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50bc15fa70 tx=0x7f50b0008040 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 msgr2=0x7f50bc154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c450 tx=0x7f50ac00c810 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 shutdown_connections 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f50a80682f0 0x7f50a806a7a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50bc161340 0x7f50bc1f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50bc0d2320 0x7f50bc154210 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 --2- 172.21.15.53:0/3531138397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50bc15c9b0 0x7f50bc143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 >> 172.21.15.53:0/3531138397 conn(0x7f50bc072600 msgr2=0x7f50bc0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 shutdown_connections 2023-11-28T11:20:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.772+0000 7f50c23ff700 1 -- 172.21.15.53:0/3531138397 wait complete. 2023-11-28T11:20:27.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:20:27.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.997+0000 7fd7ed9ff700 1 Processor -- start 2023-11-28T11:20:27.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.997+0000 7fd7ed9ff700 1 -- start start 2023-11-28T11:20:27.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7ed9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7ed9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 0x7fd7e80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:27.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7ed9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e80d1950 con 0x7fd7e8161340 2023-11-28T11:20:27.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7ed9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e80b5800 con 0x7fd7e815c9b0 2023-11-28T11:20:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7ed9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e80ce140 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7e6ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7e6ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44436/0 (socket says 172.21.15.53:44436) 2023-11-28T11:20:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.998+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/749452383 learned_addr learned my addr 172.21.15.53:0/749452383 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e77fe700 1 --2- 172.21.15.53:0/749452383 >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 0x7fd7e80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/749452383 >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 msgr2=0x7fd7e80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e6ffd700 1 --2- 172.21.15.53:0/749452383 >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 0x7fd7e80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/749452383 >> v1:172.21.15.157:6789/0 conn(0x7fd7e8161340 legacy=0x7fd7e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/749452383 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80d24f0 con 0x7fd7e815c9b0 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e77fe700 1 --2- 172.21.15.53:0/749452383 >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 0x7fd7e80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e6ffd700 1 --2- 172.21.15.53:0/749452383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806c170 tx=0x7fd7d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3a0e8b7c3d6ec2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:27.999+0000 7fd7e5ffb700 1 -- 172.21.15.53:0/749452383 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800b870 con 0x7fd7e815c9b0 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7e5ffb700 1 -- 172.21.15.53:0/749452383 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d800b9d0 con 0x7fd7e815c9b0 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7e5ffb700 1 -- 172.21.15.53:0/749452383 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800bd60 con 0x7fd7e815c9b0 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/749452383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e815c9b0 msgr2=0x7fd7e80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/749452383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806c170 tx=0x7fd7d8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/749452383 shutdown_connections 2023-11-28T11:20:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/749452383 >> v2:172.21.15.157:3300/0 conn(0x7fd7e80cd7c0 0x7fd7e80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/749452383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e815c9b0 0x7fd7e80b4bf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/749452383 >> 172.21.15.53:0/749452383 conn(0x7fd7e8072600 msgr2=0x7fd7e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/749452383 shutdown_connections 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.000+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/749452383 wait complete. 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.002+0000 7fd7ed9ff700 1 Processor -- start 2023-11-28T11:20:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.002+0000 7fd7ed9ff700 1 -- start start 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7e815c9b0 0x7fd7e81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7e8161340 0x7fd7e80b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80cd540 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81e7230 con 0x7fd7e815c9b0 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7ed9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81f2b80 con 0x7fd7e8161340 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44452/0 (socket says 172.21.15.53:44452) 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/1750928894 learned_addr learned my addr 172.21.15.53:0/1750928894 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7e67fc700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7e815c9b0 0x7fd7e81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.003+0000 7fd7e77fe700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7e8161340 0x7fd7e80b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7e8161340 msgr2=0x7fd7e80b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7e8161340 0x7fd7e80b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7e815c9b0 msgr2=0x7fd7e81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7e815c9b0 0x7fd7e81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 -- 172.21.15.53:0/1750928894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8009550 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7e6ffd700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8000c00 tx=0x7fd7d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8004310 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d801e070 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e80d1300 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80036a0 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.004+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e80b5520 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.005+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e80d15e0 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.006+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7d8029020 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.007+0000 7fd7cffff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.007+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7d8086e90 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.008+0000 7fd7e67fc700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.008+0000 7fd7e67fc700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 0x7fd7d406a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80b8d80 tx=0x7fd7dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.012+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7d8053580 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.212+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7e80745c0 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.212+0000 7fd7cffff700 1 -- 172.21.15.53:0/1750928894 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7d8056ba0 con 0x7fd7e80cd7c0 2023-11-28T11:20:28.213 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 msgr2=0x7fd7d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 0x7fd7d406a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80b8d80 tx=0x7fd7dc009040 comp rx=0 tx=0).stop 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 msgr2=0x7fd7e80cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8000c00 tx=0x7fd7d8004750 comp rx=0 tx=0).stop 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 shutdown_connections 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7e8161340 0x7fd7e80b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e80cd7c0 0x7fd7e80cd000 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 --2- 172.21.15.53:0/1750928894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7e815c9b0 0x7fd7e81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 >> 172.21.15.53:0/1750928894 conn(0x7fd7e8072600 msgr2=0x7fd7e815f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 shutdown_connections 2023-11-28T11:20:28.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.215+0000 7fd7ed9ff700 1 -- 172.21.15.53:0/1750928894 wait complete. 2023-11-28T11:20:28.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:20:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.437+0000 7f48da3ff700 1 Processor -- start 2023-11-28T11:20:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.438+0000 7f48da3ff700 1 -- start start 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48da3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f48d4070c50 0x7f48d40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48da3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d415add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48da3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d415e1a0 con 0x7f48d4158200 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48da3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d415f070 con 0x7f48d4070c50 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48da3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b0fd0 con 0x7f48d40b0bb0 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48d3fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f48d4158200 0x7f48d40caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34092/0 (socket says 172.21.15.53:34092) 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48d3fff700 1 -- 172.21.15.53:0/3551851798 learned_addr learned my addr 172.21.15.53:0/3551851798 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48d37fe700 1 --2- 172.21.15.53:0/3551851798 >> v2:172.21.15.53:3300/0 conn(0x7f48d4070c50 0x7f48d40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48d899c700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.439+0000 7f48d2ffd700 1 -- 172.21.15.53:0/3551851798 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680729017 0 0) 0x7f48d415e1a0 con 0x7f48d4158200 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d2ffd700 1 -- 172.21.15.53:0/3551851798 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48bc0035b0 con 0x7f48d4158200 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d37fe700 1 -- 172.21.15.53:0/3551851798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 msgr2=0x7f48d415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d37fe700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d415add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d37fe700 1 -- 172.21.15.53:0/3551851798 >> v1:172.21.15.53:6789/0 conn(0x7f48d4158200 legacy=0x7f48d40caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d37fe700 1 -- 172.21.15.53:0/3551851798 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40cb180 con 0x7f48d4070c50 2023-11-28T11:20:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d37fe700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d40b0670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009840 tx=0x7f48c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbeb34c1ae264ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d2ffd700 1 -- 172.21.15.53:0/3551851798 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011830 con 0x7f48d4070c50 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d2ffd700 1 -- 172.21.15.53:0/3551851798 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8011990 con 0x7f48d4070c50 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.440+0000 7f48d2ffd700 1 -- 172.21.15.53:0/3551851798 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011d20 con 0x7f48d4070c50 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 -- 172.21.15.53:0/3551851798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 msgr2=0x7f48d40b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d40b0670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009840 tx=0x7f48c8009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 -- 172.21.15.53:0/3551851798 shutdown_connections 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 --2- 172.21.15.53:0/3551851798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d40b0670 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 -- 172.21.15.53:0/3551851798 >> 172.21.15.53:0/3551851798 conn(0x7f48d40cb670 msgr2=0x7f48d40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 -- 172.21.15.53:0/3551851798 shutdown_connections 2023-11-28T11:20:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.441+0000 7f48da3ff700 1 -- 172.21.15.53:0/3551851798 wait complete. 2023-11-28T11:20:28.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.443+0000 7f48da3ff700 1 Processor -- start 2023-11-28T11:20:28.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.443+0000 7f48da3ff700 1 -- start start 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d41f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48d4158200 0x7f48d40b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d415ae00 con 0x7f48d4070c50 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b37b0 con 0x7f48d4158200 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48da3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41f2d50 con 0x7f48d40b0bb0 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44468/0 (socket says 172.21.15.53:44468) 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 -- 172.21.15.53:0/2700388753 learned_addr learned my addr 172.21.15.53:0/2700388753 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d899c700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48d4158200 0x7f48d40b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d37fe700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d41f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 -- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 msgr2=0x7f48d41f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d41f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 -- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48d4158200 msgr2=0x7f48d40b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48d4158200 0x7f48d40b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.444+0000 7f48d3fff700 1 -- 172.21.15.53:0/2700388753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80094a0 con 0x7f48d4070c50 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48d3fff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48c4001eb0 tx=0x7f48c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005d70 con 0x7f48d4070c50 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40070d0 con 0x7f48d4070c50 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4007460 con 0x7f48d4070c50 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d41585d0 con 0x7f48d4070c50 2023-11-28T11:20:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.445+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d40710f0 con 0x7f48d4070c50 2023-11-28T11:20:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.446+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d415e3f0 con 0x7f48d4070c50 2023-11-28T11:20:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.447+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48c4019030 con 0x7f48d4070c50 2023-11-28T11:20:28.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.448+0000 7f48d0ff9700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.448+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48c4085090 con 0x7f48d4070c50 2023-11-28T11:20:28.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.449+0000 7f48d37fe700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.449+0000 7f48d37fe700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 0x7f48c006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.452+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48c40518f0 con 0x7f48d4070c50 2023-11-28T11:20:28.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.653+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48d406fd60 con 0x7f48d4070c50 2023-11-28T11:20:28.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.654+0000 7f48d0ff9700 1 -- 172.21.15.53:0/2700388753 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48c4054f10 con 0x7f48d4070c50 2023-11-28T11:20:28.655 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:20:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 msgr2=0x7f48c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 0x7f48c006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c8008030 comp rx=0 tx=0).stop 2023-11-28T11:20:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 msgr2=0x7f48d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48c4001eb0 tx=0x7f48c400c860 comp rx=0 tx=0).stop 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 shutdown_connections 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48d4158200 0x7f48d40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4070c50 0x7f48d406d3f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 --2- 172.21.15.53:0/2700388753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48d40b0bb0 0x7f48d41f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 >> 172.21.15.53:0/2700388753 conn(0x7f48d40cb670 msgr2=0x7f48d40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 shutdown_connections 2023-11-28T11:20:28.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.656+0000 7f48da3ff700 1 -- 172.21.15.53:0/2700388753 wait complete. 2023-11-28T11:20:28.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:20:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.877+0000 7f5045fff700 1 Processor -- start 2023-11-28T11:20:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.877+0000 7f5045fff700 1 -- start start 2023-11-28T11:20:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f5045fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5040161340 0x7f50400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f5045fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50400d2320 0x7f50400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f5045fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400d1950 con 0x7f504015c9b0 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f5045fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50400b5800 con 0x7f5040161340 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f5045fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50400b5300 con 0x7f50400d2320 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f503f7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f504015c9b0 0x7f50400b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34108/0 (socket says 172.21.15.53:34108) 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f503effd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5040161340 0x7f50400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.878+0000 7f503f7fe700 1 -- 172.21.15.53:0/4185097291 learned_addr learned my addr 172.21.15.53:0/4185097291 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503ffff700 1 --2- 172.21.15.53:0/4185097291 >> v2:172.21.15.157:3300/0 conn(0x7f50400d2320 0x7f50400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503effd700 1 -- 172.21.15.53:0/4185097291 >> v2:172.21.15.157:3300/0 conn(0x7f50400d2320 msgr2=0x7f50400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503effd700 1 --2- 172.21.15.53:0/4185097291 >> v2:172.21.15.157:3300/0 conn(0x7f50400d2320 0x7f50400cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503effd700 1 -- 172.21.15.53:0/4185097291 >> v1:172.21.15.53:6789/0 conn(0x7f504015c9b0 legacy=0x7f50400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503effd700 1 -- 172.21.15.53:0/4185097291 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400b5520 con 0x7f5040161340 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503e7fc700 1 -- 172.21.15.53:0/4185097291 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643645941 0 0) 0x7f50400d1950 con 0x7f504015c9b0 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503effd700 1 --2- 172.21.15.53:0/4185097291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040161340 0x7f50400d1de0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f504006c170 tx=0x7f5034009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e720bf829441bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.879+0000 7f503e7fc700 1 -- 172.21.15.53:0/4185097291 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034011830 con 0x7f5040161340 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f503e7fc700 1 -- 172.21.15.53:0/4185097291 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034011990 con 0x7f5040161340 2023-11-28T11:20:28.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f503e7fc700 1 -- 172.21.15.53:0/4185097291 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034011d20 con 0x7f5040161340 2023-11-28T11:20:28.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 -- 172.21.15.53:0/4185097291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040161340 msgr2=0x7f50400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 --2- 172.21.15.53:0/4185097291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040161340 0x7f50400d1de0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f504006c170 tx=0x7f5034009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:28.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 -- 172.21.15.53:0/4185097291 shutdown_connections 2023-11-28T11:20:28.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 --2- 172.21.15.53:0/4185097291 >> v2:172.21.15.157:3300/0 conn(0x7f50400d2320 0x7f50400cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 --2- 172.21.15.53:0/4185097291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040161340 0x7f50400d1de0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 -- 172.21.15.53:0/4185097291 >> 172.21.15.53:0/4185097291 conn(0x7f5040072600 msgr2=0x7f50400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 -- 172.21.15.53:0/4185097291 shutdown_connections 2023-11-28T11:20:28.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.880+0000 7f5045fff700 1 -- 172.21.15.53:0/4185097291 wait complete. 2023-11-28T11:20:28.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.883+0000 7f5045fff700 1 Processor -- start 2023-11-28T11:20:28.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.883+0000 7f5045fff700 1 -- start start 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 0x7f50401e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 0x7f504015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50401e76c0 con 0x7f504015c9b0 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c9ab0 con 0x7f50400d2320 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f5045fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50401e6ab0 con 0x7f5040161340 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f503effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f503effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44490/0 (socket says 172.21.15.53:44490) 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f503effd700 1 -- 172.21.15.53:0/1394867734 learned_addr learned my addr 172.21.15.53:0/1394867734 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.884+0000 7f503f7fe700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 0x7f50401e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503ffff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 0x7f504015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 -- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 msgr2=0x7f504015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 0x7f504015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 -- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 msgr2=0x7f50401e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 0x7f50401e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 -- 172.21.15.53:0/1394867734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340094a0 con 0x7f504015c9b0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503f7fe700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 0x7f50401e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503ffff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 0x7f504015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f503effd700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f50340097c0 tx=0x7f5034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50340038a0 con 0x7f504015c9b0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503401f070 con 0x7f504015c9b0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034003c40 con 0x7f504015c9b0 2023-11-28T11:20:28.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.885+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400d15e0 con 0x7f504015c9b0 2023-11-28T11:20:28.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.886+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50400d1300 con 0x7f504015c9b0 2023-11-28T11:20:28.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.887+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504015cd80 con 0x7f504015c9b0 2023-11-28T11:20:28.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.887+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f503400f050 con 0x7f504015c9b0 2023-11-28T11:20:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.888+0000 7f5027fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 0x7f502c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.888+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5034021070 con 0x7f504015c9b0 2023-11-28T11:20:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.889+0000 7f503f7fe700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 0x7f502c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.889+0000 7f503f7fe700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 0x7f502c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:28.893+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5034053f10 con 0x7f504015c9b0 2023-11-28T11:20:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.114+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50400745c0 con 0x7f504015c9b0 2023-11-28T11:20:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.115+0000 7f5027fff700 1 -- 172.21.15.53:0/1394867734 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5034057530 con 0x7f504015c9b0 2023-11-28T11:20:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 msgr2=0x7f502c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 0x7f502c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030007d30 comp rx=0 tx=0).stop 2023-11-28T11:20:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 msgr2=0x7f50400c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f50340097c0 tx=0x7f5034004750 comp rx=0 tx=0).stop 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 shutdown_connections 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f502c0682f0 0x7f502c06a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5040161340 0x7f504015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400d2320 0x7f50401e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 --2- 172.21.15.53:0/1394867734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f504015c9b0 0x7f50400c9570 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 >> 172.21.15.53:0/1394867734 conn(0x7f5040072600 msgr2=0x7f50400b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 shutdown_connections 2023-11-28T11:20:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.117+0000 7f5045fff700 1 -- 172.21.15.53:0/1394867734 wait complete. 2023-11-28T11:20:29.120 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:20:29.136 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:20:29.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:29.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:20:29.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:20:29.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:29.244 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:20:29.297 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:20:29.298 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:29.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:20:29.344 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:29.344 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:29.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:20:29.388 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:20:29.388 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:29.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:20:29.434 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.634+0000 7fd2e3bff700 1 Processor -- start 2023-11-28T11:20:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.634+0000 7fd2e3bff700 1 -- start start 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.635+0000 7fd2e3bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2dc15c9b0 0x7fd2dc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.635+0000 7fd2e3bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2dc161340 0x7fd2dc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.635+0000 7fd2e3bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0d1950 con 0x7fd2dc0d2320 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.635+0000 7fd2e3bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b5800 con 0x7fd2dc161340 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.635+0000 7fd2e3bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b5520 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2dc161340 0x7fd2dc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2dc161340 0x7fd2dc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44504/0 (socket says 172.21.15.53:44504) 2023-11-28T11:20:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e219c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd2dc0d2320 0x7fd2dc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34120/0 (socket says 172.21.15.53:34120) 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 -- 172.21.15.53:0/620049834 learned_addr learned my addr 172.21.15.53:0/620049834 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e199b700 1 --2- 172.21.15.53:0/620049834 >> v2:172.21.15.157:3300/0 conn(0x7fd2dc15c9b0 0x7fd2dc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 -- 172.21.15.53:0/620049834 >> v2:172.21.15.157:3300/0 conn(0x7fd2dc15c9b0 msgr2=0x7fd2dc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 --2- 172.21.15.53:0/620049834 >> v2:172.21.15.157:3300/0 conn(0x7fd2dc15c9b0 0x7fd2dc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 -- 172.21.15.53:0/620049834 >> v1:172.21.15.53:6789/0 conn(0x7fd2dc0d2320 legacy=0x7fd2dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 -- 172.21.15.53:0/620049834 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0d1300 con 0x7fd2dc161340 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e0999700 1 -- 172.21.15.53:0/620049834 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778975979 0 0) 0x7fd2dc0d1950 con 0x7fd2dc0d2320 2023-11-28T11:20:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.636+0000 7fd2e119a700 1 --2- 172.21.15.53:0/620049834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc161340 0x7fd2dc0d1de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc067460 tx=0x7fd2cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eb347605d648c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:29.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e0999700 1 -- 172.21.15.53:0/620049834 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011830 con 0x7fd2dc161340 2023-11-28T11:20:29.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e0999700 1 -- 172.21.15.53:0/620049834 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc011990 con 0x7fd2dc161340 2023-11-28T11:20:29.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e0999700 1 -- 172.21.15.53:0/620049834 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011d20 con 0x7fd2dc161340 2023-11-28T11:20:29.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 -- 172.21.15.53:0/620049834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc161340 msgr2=0x7fd2dc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/620049834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc161340 0x7fd2dc0d1de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc067460 tx=0x7fd2cc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 -- 172.21.15.53:0/620049834 shutdown_connections 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/620049834 >> v2:172.21.15.157:3300/0 conn(0x7fd2dc15c9b0 0x7fd2dc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/620049834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc161340 0x7fd2dc0d1de0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 -- 172.21.15.53:0/620049834 >> 172.21.15.53:0/620049834 conn(0x7fd2dc072600 msgr2=0x7fd2dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 -- 172.21.15.53:0/620049834 shutdown_connections 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.637+0000 7fd2e3bff700 1 -- 172.21.15.53:0/620049834 wait complete. 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.639+0000 7fd2e3bff700 1 Processor -- start 2023-11-28T11:20:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.639+0000 7fd2e3bff700 1 -- start start 2023-11-28T11:20:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2dc0d2320 0x7fd2dc152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 0x7fd2dc154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1f3030 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0c82f0 con 0x7fd2dc161340 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0c8e70 con 0x7fd2dc0d2320 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.640+0000 7fd2e119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44520/0 (socket says 172.21.15.53:44520) 2023-11-28T11:20:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 -- 172.21.15.53:0/2186293745 learned_addr learned my addr 172.21.15.53:0/2186293745 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e199b700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2dc0d2320 0x7fd2dc152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e219c700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 0x7fd2dc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 -- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2dc0d2320 msgr2=0x7fd2dc152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2dc0d2320 0x7fd2dc152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 -- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 msgr2=0x7fd2dc154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 0x7fd2dc154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 -- 172.21.15.53:0/2186293745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc009550 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e219c700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 0x7fd2dc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2e119a700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0675d0 tx=0x7fd2cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.641+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004590 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.642+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc0036a0 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.642+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc0b5130 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.642+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc003a30 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.642+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc0d15e0 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.643+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc0b52c0 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.643+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2cc02d070 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.644+0000 7fd2d27fc700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 0x7fd2c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.644+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2cc021030 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.645+0000 7fd2e199b700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 0x7fd2c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.645+0000 7fd2e199b700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 0x7fd2c006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009be0 tx=0x7fd2d8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:29.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.648+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2cc053bd0 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.899+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2dc0c8660 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.900+0000 7fd2d27fc700 1 -- 172.21.15.53:0/2186293745 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fd2cc015090 con 0x7fd2dc15c9b0 2023-11-28T11:20:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:29.901 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:20:29.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 msgr2=0x7fd2c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 0x7fd2c006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009be0 tx=0x7fd2d8009210 comp rx=0 tx=0).stop 2023-11-28T11:20:29.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 msgr2=0x7fd2dc1f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0675d0 tx=0x7fd2cc004750 comp rx=0 tx=0).stop 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 shutdown_connections 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd2c00682f0 0x7fd2c006a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2dc0d2320 0x7fd2dc152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2dc161340 0x7fd2dc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 --2- 172.21.15.53:0/2186293745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc15c9b0 0x7fd2dc1f2af0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 >> 172.21.15.53:0/2186293745 conn(0x7fd2dc072600 msgr2=0x7fd2dc160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 shutdown_connections 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:29.902+0000 7fd2e3bff700 1 -- 172.21.15.53:0/2186293745 wait complete. 2023-11-28T11:20:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:20:29.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.122+0000 7fc5339ff700 1 Processor -- start 2023-11-28T11:20:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.122+0000 7fc5339ff700 1 -- start start 2023-11-28T11:20:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc5339ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 0x7fc52c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc5339ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0d1950 con 0x7fc52c161340 2023-11-28T11:20:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc5339ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0ce140 con 0x7fc52c15c9b0 2023-11-28T11:20:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc5339ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b5800 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc530f9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc52c161340 0x7fc52c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34136/0 (socket says 172.21.15.53:34136) 2023-11-28T11:20:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.123+0000 7fc530f9a700 1 -- 172.21.15.53:0/1631528155 learned_addr learned my addr 172.21.15.53:0/1631528155 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc531f9c700 1 --2- 172.21.15.53:0/1631528155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 0x7fc52c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810792085 0 0) 0x7fc52c0d1950 con 0x7fc52c161340 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5180035b0 con 0x7fc52c161340 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105609003 0 0) 0x7fc52c0ce140 con 0x7fc52c15c9b0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c0d1950 con 0x7fc52c15c9b0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc531f9c700 1 -- 172.21.15.53:0/1631528155 >> v1:172.21.15.157:6789/0 conn(0x7fc52c15c9b0 legacy=0x7fc52c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc531f9c700 1 -- 172.21.15.53:0/1631528155 >> v1:172.21.15.53:6789/0 conn(0x7fc52c161340 legacy=0x7fc52c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.124+0000 7fc531f9c700 1 -- 172.21.15.53:0/1631528155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0d24f0 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc531f9c700 1 --2- 172.21.15.53:0/1631528155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 0x7fc52c0cdc00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc524009dc0 tx=0x7fc52400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd6ebd40eceffc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400e070 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400bb20 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc523fff700 1 -- 172.21.15.53:0/1631528155 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524012430 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc5339ff700 1 -- 172.21.15.53:0/1631528155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 msgr2=0x7fc52c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc5339ff700 1 --2- 172.21.15.53:0/1631528155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 0x7fc52c0cdc00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc524009dc0 tx=0x7fc52400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc5339ff700 1 -- 172.21.15.53:0/1631528155 shutdown_connections 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc5339ff700 1 --2- 172.21.15.53:0/1631528155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c0cd7c0 0x7fc52c0cdc00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.125+0000 7fc5339ff700 1 -- 172.21.15.53:0/1631528155 >> 172.21.15.53:0/1631528155 conn(0x7fc52c072600 msgr2=0x7fc52c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.126+0000 7fc5339ff700 1 -- 172.21.15.53:0/1631528155 shutdown_connections 2023-11-28T11:20:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.126+0000 7fc5339ff700 1 -- 172.21.15.53:0/1631528155 wait complete. 2023-11-28T11:20:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.127+0000 7fc5339ff700 1 Processor -- start 2023-11-28T11:20:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.127+0000 7fc5339ff700 1 -- start start 2023-11-28T11:20:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c0cd7c0 0x7fc52c157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c15c9b0 0x7fc52c1f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0cc440 con 0x7fc52c161340 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0cb7c0 con 0x7fc52c0cd7c0 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.128+0000 7fc5339ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0cbaa0 con 0x7fc52c15c9b0 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44534/0 (socket says 172.21.15.53:44534) 2023-11-28T11:20:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 -- 172.21.15.53:0/4117552019 learned_addr learned my addr 172.21.15.53:0/4117552019 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc53179b700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c0cd7c0 0x7fc52c157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc530f9a700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c15c9b0 0x7fc52c1f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 -- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c15c9b0 msgr2=0x7fc52c1f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c15c9b0 0x7fc52c1f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 -- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c0cd7c0 msgr2=0x7fc52c157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c0cd7c0 0x7fc52c157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 -- 172.21.15.53:0/4117552019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524009a70 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.129+0000 7fc531f9c700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b370 tx=0x7fc524004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.130+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52401b810 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.130+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524020070 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.130+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52401bbf0 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.130+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c0d1300 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.130+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c0b5520 con 0x7fc52c161340 2023-11-28T11:20:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.131+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c0d15e0 con 0x7fc52c161340 2023-11-28T11:20:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.132+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc52401bd50 con 0x7fc52c161340 2023-11-28T11:20:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.133+0000 7fc521ffb700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 0x7fc51006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.133+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc524022070 con 0x7fc52c161340 2023-11-28T11:20:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.133+0000 7fc53179b700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 0x7fc51006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.134+0000 7fc53179b700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 0x7fc51006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc528007fd0 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.137+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc524054b80 con 0x7fc52c161340 2023-11-28T11:20:30.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.342+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc52c0cc580 con 0x7fc52c161340 2023-11-28T11:20:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.343+0000 7fc521ffb700 1 -- 172.21.15.53:0/4117552019 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17979 (secure 0 0 0) 0x7fc52c0cc580 con 0x7fc52c161340 2023-11-28T11:20:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:30.344 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:45.403641+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 msgr2=0x7fc51006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 0x7fc51006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc528007fd0 tx=0x7fc528009040 comp rx=0 tx=0).stop 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 msgr2=0x7fc52c0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b370 tx=0x7fc524004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 shutdown_connections 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc5100682f0 0x7fc51006a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc52c161340 0x7fc52c0cac00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc52c0cd7c0 0x7fc52c157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 --2- 172.21.15.53:0/4117552019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc52c15c9b0 0x7fc52c1f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 >> 172.21.15.53:0/4117552019 conn(0x7fc52c072600 msgr2=0x7fc52c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 shutdown_connections 2023-11-28T11:20:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.345+0000 7fc5339ff700 1 -- 172.21.15.53:0/4117552019 wait complete. 2023-11-28T11:20:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:19:42.218381+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:19:43.389004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:20:30.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:20:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.570+0000 7ffb489ff700 1 Processor -- start 2023-11-28T11:20:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.571+0000 7ffb489ff700 1 -- start start 2023-11-28T11:20:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb489ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 0x7ffb440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb489ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 0x7ffb440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb489ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb440d1950 con 0x7ffb440d2320 2023-11-28T11:20:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb489ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb440b5800 con 0x7ffb4415c9b0 2023-11-28T11:20:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb489ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440b5520 con 0x7ffb44161340 2023-11-28T11:20:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb3bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 0x7ffb440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb42d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffb440d2320 0x7ffb440cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:51144/0 (socket says 172.21.15.53:51144) 2023-11-28T11:20:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb42d9d700 1 -- 172.21.15.53:0/2176088480 learned_addr learned my addr 172.21.15.53:0/2176088480 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.572+0000 7ffb4259c700 1 --2- 172.21.15.53:0/2176088480 >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 0x7ffb440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb41d9b700 1 -- 172.21.15.53:0/2176088480 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490201608 0 0) 0x7ffb440d1950 con 0x7ffb440d2320 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb41d9b700 1 -- 172.21.15.53:0/2176088480 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb300035b0 con 0x7ffb440d2320 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb3bfff700 1 -- 172.21.15.53:0/2176088480 >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 msgr2=0x7ffb440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb3bfff700 1 --2- 172.21.15.53:0/2176088480 >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 0x7ffb440b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb3bfff700 1 -- 172.21.15.53:0/2176088480 >> v1:172.21.15.157:6789/0 conn(0x7ffb440d2320 legacy=0x7ffb440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb3bfff700 1 -- 172.21.15.53:0/2176088480 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440d1300 con 0x7ffb44161340 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb4259c700 1 --2- 172.21.15.53:0/2176088480 >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 0x7ffb440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.573+0000 7ffb3bfff700 1 --2- 172.21.15.53:0/2176088480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 0x7ffb440d1de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009840 tx=0x7ffb2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd490a385762c12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb41d9b700 1 -- 172.21.15.53:0/2176088480 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c011830 con 0x7ffb44161340 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb41d9b700 1 -- 172.21.15.53:0/2176088480 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c011990 con 0x7ffb44161340 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb41d9b700 1 -- 172.21.15.53:0/2176088480 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c011d20 con 0x7ffb44161340 2023-11-28T11:20:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 -- 172.21.15.53:0/2176088480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 msgr2=0x7ffb440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 --2- 172.21.15.53:0/2176088480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 0x7ffb440d1de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009840 tx=0x7ffb2c009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 -- 172.21.15.53:0/2176088480 shutdown_connections 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 --2- 172.21.15.53:0/2176088480 >> v2:172.21.15.157:3300/0 conn(0x7ffb4415c9b0 0x7ffb440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 --2- 172.21.15.53:0/2176088480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44161340 0x7ffb440d1de0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 -- 172.21.15.53:0/2176088480 >> 172.21.15.53:0/2176088480 conn(0x7ffb44072600 msgr2=0x7ffb440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 -- 172.21.15.53:0/2176088480 shutdown_connections 2023-11-28T11:20:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.574+0000 7ffb489ff700 1 -- 172.21.15.53:0/2176088480 wait complete. 2023-11-28T11:20:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.576+0000 7ffb489ff700 1 Processor -- start 2023-11-28T11:20:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.576+0000 7ffb489ff700 1 -- start start 2023-11-28T11:20:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb440d2320 0x7ffb4414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb4415c9b0 0x7ffb4415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4415e210 con 0x7ffb44161340 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4415e4f0 con 0x7ffb440d2320 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb489ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4415e7d0 con 0x7ffb4415c9b0 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb4259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb440d2320 0x7ffb4414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44536/0 (socket says 172.21.15.53:44536) 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb42d9d700 1 -- 172.21.15.53:0/1645107479 learned_addr learned my addr 172.21.15.53:0/1645107479 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.577+0000 7ffb3bfff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb4415c9b0 0x7ffb4415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 -- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb4415c9b0 msgr2=0x7ffb4415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb4415c9b0 0x7ffb4415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 -- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb440d2320 msgr2=0x7ffb4414de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb440d2320 0x7ffb4414de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 -- 172.21.15.53:0/1645107479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340094a0 con 0x7ffb44161340 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb42d9d700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00caa0 tx=0x7ffb3c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00a9e0 con 0x7ffb44161340 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00ab40 con 0x7ffb44161340 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c012400 con 0x7ffb44161340 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.578+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c0094a0 con 0x7ffb44161340 2023-11-28T11:20:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.579+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb440b5130 con 0x7ffb44161340 2023-11-28T11:20:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.580+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb440d15e0 con 0x7ffb44161340 2023-11-28T11:20:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.581+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb3c019030 con 0x7ffb44161340 2023-11-28T11:20:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.581+0000 7ffb3affd700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 0x7ffb2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.582+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb3c085740 con 0x7ffb44161340 2023-11-28T11:20:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.582+0000 7ffb4259c700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 0x7ffb2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.583+0000 7ffb4259c700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 0x7ffb2406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffb440ce3a0 tx=0x7ffb34009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.586+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb3c052060 con 0x7ffb44161340 2023-11-28T11:20:30.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.785+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb440b52c0 con 0x7ffb44161340 2023-11-28T11:20:30.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.786+0000 7ffb3affd700 1 -- 172.21.15.53:0/1645107479 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb3c055680 con 0x7ffb44161340 2023-11-28T11:20:30.787 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 msgr2=0x7ffb2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 0x7ffb2406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffb440ce3a0 tx=0x7ffb34009210 comp rx=0 tx=0).stop 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 msgr2=0x7ffb4414d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00caa0 tx=0x7ffb3c00cdb0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 shutdown_connections 2023-11-28T11:20:30.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb240682f0 0x7ffb2406a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb44161340 0x7ffb4414d6b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb440d2320 0x7ffb4414de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 --2- 172.21.15.53:0/1645107479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb4415c9b0 0x7ffb4415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 >> 172.21.15.53:0/1645107479 conn(0x7ffb44072600 msgr2=0x7ffb4415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 shutdown_connections 2023-11-28T11:20:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:30.788+0000 7ffb489ff700 1 -- 172.21.15.53:0/1645107479 wait complete. 2023-11-28T11:20:30.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:20:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.011+0000 7ff0509ff700 1 Processor -- start 2023-11-28T11:20:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.011+0000 7ff0509ff700 1 -- start start 2023-11-28T11:20:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.012+0000 7ff0509ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff04c15c9b0 0x7ff04c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.012+0000 7ff0509ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff04c161340 0x7ff04c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.012+0000 7ff0509ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c0d1950 con 0x7ff04c0d2320 2023-11-28T11:20:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.012+0000 7ff0509ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0b5800 con 0x7ff04c161340 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.012+0000 7ff0509ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0b5520 con 0x7ff04c15c9b0 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff04c161340 0x7ff04c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff04c161340 0x7ff04c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44654/0 (socket says 172.21.15.53:44654) 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 -- 172.21.15.53:0/2844208139 learned_addr learned my addr 172.21.15.53:0/2844208139 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff04a59c700 1 --2- 172.21.15.53:0/2844208139 >> v2:172.21.15.157:3300/0 conn(0x7ff04c15c9b0 0x7ff04c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 -- 172.21.15.53:0/2844208139 >> v2:172.21.15.157:3300/0 conn(0x7ff04c15c9b0 msgr2=0x7ff04c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 --2- 172.21.15.53:0/2844208139 >> v2:172.21.15.157:3300/0 conn(0x7ff04c15c9b0 0x7ff04c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 -- 172.21.15.53:0/2844208139 >> v1:172.21.15.157:6789/0 conn(0x7ff04c0d2320 legacy=0x7ff04c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 -- 172.21.15.53:0/2844208139 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0d1300 con 0x7ff04c161340 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.013+0000 7ff049d9b700 1 --2- 172.21.15.53:0/2844208139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c0d1de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff04c067460 tx=0x7ff040009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd1c8b0ce5a5b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff04959a700 1 -- 172.21.15.53:0/2844208139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000b870 con 0x7ff04c161340 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff04959a700 1 -- 172.21.15.53:0/2844208139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000b9d0 con 0x7ff04c161340 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff04959a700 1 -- 172.21.15.53:0/2844208139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000bd60 con 0x7ff04c161340 2023-11-28T11:20:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 -- 172.21.15.53:0/2844208139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 msgr2=0x7ff04c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 --2- 172.21.15.53:0/2844208139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c0d1de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff04c067460 tx=0x7ff040009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 -- 172.21.15.53:0/2844208139 shutdown_connections 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 --2- 172.21.15.53:0/2844208139 >> v2:172.21.15.157:3300/0 conn(0x7ff04c15c9b0 0x7ff04c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 --2- 172.21.15.53:0/2844208139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c0d1de0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 -- 172.21.15.53:0/2844208139 >> 172.21.15.53:0/2844208139 conn(0x7ff04c072600 msgr2=0x7ff04c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 -- 172.21.15.53:0/2844208139 shutdown_connections 2023-11-28T11:20:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.014+0000 7ff0509ff700 1 -- 172.21.15.53:0/2844208139 wait complete. 2023-11-28T11:20:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.016+0000 7ff0509ff700 1 Processor -- start 2023-11-28T11:20:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.016+0000 7ff0509ff700 1 -- start start 2023-11-28T11:20:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0d2320 0x7ff04c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 0x7ff04c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1f2840 con 0x7ff04c161340 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0cb330 con 0x7ff04c0d2320 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff0509ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0c9250 con 0x7ff04c15c9b0 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff04ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff04ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44670/0 (socket says 172.21.15.53:44670) 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.017+0000 7ff04ad9d700 1 -- 172.21.15.53:0/1522926782 learned_addr learned my addr 172.21.15.53:0/1522926782 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff049d9b700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 0x7ff04c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04a59c700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0d2320 0x7ff04c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 -- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 msgr2=0x7ff04c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 0x7ff04c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 -- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0d2320 msgr2=0x7ff04c0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0d2320 0x7ff04c0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 -- 172.21.15.53:0/1522926782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009550 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff049d9b700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 0x7ff04c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff04ad9d700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff04400cbc0 tx=0x7ff04400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044003c00 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044003d60 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400ab90 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c0b5130 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.018+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c0d15e0 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.020+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0b52c0 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.021+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff044010050 con 0x7ff04c161340 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.021+0000 7ff03affd700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 0x7ff02c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.021+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff044085c40 con 0x7ff04c161340 2023-11-28T11:20:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.022+0000 7ff04a59c700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 0x7ff02c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.022+0000 7ff04a59c700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 0x7ff02c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009930 tx=0x7ff03c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.025+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0440523b0 con 0x7ff04c161340 2023-11-28T11:20:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.228+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff04c0745c0 con 0x7ff04c161340 2023-11-28T11:20:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.228+0000 7ff03affd700 1 -- 172.21.15.53:0/1522926782 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0440559d0 con 0x7ff04c161340 2023-11-28T11:20:31.229 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 msgr2=0x7ff02c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 0x7ff02c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009930 tx=0x7ff03c008040 comp rx=0 tx=0).stop 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 msgr2=0x7ff04c14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff04400cbc0 tx=0x7ff04400cf80 comp rx=0 tx=0).stop 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 shutdown_connections 2023-11-28T11:20:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff02c0682f0 0x7ff02c06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff04c161340 0x7ff04c14c580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0d2320 0x7ff04c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 --2- 172.21.15.53:0/1522926782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04c15c9b0 0x7ff04c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 >> 172.21.15.53:0/1522926782 conn(0x7ff04c072600 msgr2=0x7ff04c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 shutdown_connections 2023-11-28T11:20:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.231+0000 7ff0509ff700 1 -- 172.21.15.53:0/1522926782 wait complete. 2023-11-28T11:20:31.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:20:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.454+0000 7ffbbf5ff700 1 Processor -- start 2023-11-28T11:20:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.454+0000 7ffbbf5ff700 1 -- start start 2023-11-28T11:20:31.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbf5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffbb8161340 0x7ffbb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbf5ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbb80d2320 0x7ffbb80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbf5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80d1950 con 0x7ffbb815c9b0 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbf5ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b5800 con 0x7ffbb80d2320 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbf5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b5300 con 0x7ffbb8161340 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbd39b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ffbb815c9b0 0x7ffbb80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38118/0 (socket says 172.21.15.53:38118) 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbd39b700 1 -- 172.21.15.53:0/1110903639 learned_addr learned my addr 172.21.15.53:0/1110903639 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbdb9c700 1 --2- 172.21.15.53:0/1110903639 >> v2:172.21.15.53:3300/0 conn(0x7ffbb80d2320 0x7ffbb80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.455+0000 7ffbbcb9a700 1 --2- 172.21.15.53:0/1110903639 >> v2:172.21.15.157:3300/0 conn(0x7ffbb8161340 0x7ffbb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbaffff700 1 -- 172.21.15.53:0/1110903639 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059928947 0 0) 0x7ffbb80d1950 con 0x7ffbb815c9b0 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbbdb9c700 1 -- 172.21.15.53:0/1110903639 >> v2:172.21.15.157:3300/0 conn(0x7ffbb8161340 msgr2=0x7ffbb80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbbdb9c700 1 --2- 172.21.15.53:0/1110903639 >> v2:172.21.15.157:3300/0 conn(0x7ffbb8161340 0x7ffbb80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbbdb9c700 1 -- 172.21.15.53:0/1110903639 >> v1:172.21.15.53:6789/0 conn(0x7ffbb815c9b0 legacy=0x7ffbb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbbdb9c700 1 -- 172.21.15.53:0/1110903639 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80b5520 con 0x7ffbb80d2320 2023-11-28T11:20:31.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbbdb9c700 1 --2- 172.21.15.53:0/1110903639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb80d2320 0x7ffbb80cd7c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80685c0 tx=0x7ffbb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7f6d24dbb613a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbaffff700 1 -- 172.21.15.53:0/1110903639 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000e070 con 0x7ffbb80d2320 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbaffff700 1 -- 172.21.15.53:0/1110903639 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000bae0 con 0x7ffbb80d2320 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.456+0000 7ffbaffff700 1 -- 172.21.15.53:0/1110903639 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000be90 con 0x7ffbb80d2320 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1110903639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb80d2320 msgr2=0x7ffbb80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1110903639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb80d2320 0x7ffbb80cd7c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80685c0 tx=0x7ffbb000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1110903639 shutdown_connections 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1110903639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb80d2320 0x7ffbb80cd7c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1110903639 >> v2:172.21.15.157:3300/0 conn(0x7ffbb8161340 0x7ffbb80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1110903639 >> 172.21.15.53:0/1110903639 conn(0x7ffbb8072600 msgr2=0x7ffbb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1110903639 shutdown_connections 2023-11-28T11:20:31.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.457+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1110903639 wait complete. 2023-11-28T11:20:31.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.459+0000 7ffbbf5ff700 1 Processor -- start 2023-11-28T11:20:31.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.459+0000 7ffbbf5ff700 1 -- start start 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 0x7ffbb80c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbb8161340 0x7ffbb80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c8b80 con 0x7ffbb815c9b0 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c81b0 con 0x7ffbb8161340 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbf5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c6510 con 0x7ffbb80d2320 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44686/0 (socket says 172.21.15.53:44686) 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 -- 172.21.15.53:0/1160925260 learned_addr learned my addr 172.21.15.53:0/1160925260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 -- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 msgr2=0x7ffbb80c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbd39b700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 0x7ffbb80c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbdb9c700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbb8161340 0x7ffbb80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 0x7ffbb80c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 -- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbb8161340 msgr2=0x7ffbb80c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbb8161340 0x7ffbb80c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbcb9a700 1 -- 172.21.15.53:0/1160925260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0009a70 con 0x7ffbb815c9b0 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.460+0000 7ffbbd39b700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 0x7ffbb80c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbbcb9a700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffba800e9e0 tx=0x7ffba800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8009710 con 0x7ffbb815c9b0 2023-11-28T11:20:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800c850 con 0x7ffbb815c9b0 2023-11-28T11:20:31.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb80d15e0 con 0x7ffbb815c9b0 2023-11-28T11:20:31.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800cbc0 con 0x7ffbb815c9b0 2023-11-28T11:20:31.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.461+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80d1300 con 0x7ffbb815c9b0 2023-11-28T11:20:31.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.462+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb815cd80 con 0x7ffbb815c9b0 2023-11-28T11:20:31.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.463+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffba8019030 con 0x7ffbb815c9b0 2023-11-28T11:20:31.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.465+0000 7ffbadffb700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 0x7ffb9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.465+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffba8085d40 con 0x7ffbb815c9b0 2023-11-28T11:20:31.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.465+0000 7ffbbd39b700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 0x7ffb9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.466+0000 7ffbbd39b700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 0x7ffb9c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007fd0 tx=0x7ffbb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.469+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffba80524b0 con 0x7ffbb815c9b0 2023-11-28T11:20:31.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.670+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbb8066d60 con 0x7ffbb815c9b0 2023-11-28T11:20:31.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.670+0000 7ffbadffb700 1 -- 172.21.15.53:0/1160925260 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffba8017020 con 0x7ffbb815c9b0 2023-11-28T11:20:31.672 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 msgr2=0x7ffb9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 0x7ffb9c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007fd0 tx=0x7ffbb4009040 comp rx=0 tx=0).stop 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 msgr2=0x7ffbb80c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffba800e9e0 tx=0x7ffba800eda0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 shutdown_connections 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ffb9c0682f0 0x7ffb9c06a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbb80d2320 0x7ffbb80c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbb8161340 0x7ffbb80c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 --2- 172.21.15.53:0/1160925260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb815c9b0 0x7ffbb80c5fd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 >> 172.21.15.53:0/1160925260 conn(0x7ffbb8072600 msgr2=0x7ffbb8160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 shutdown_connections 2023-11-28T11:20:31.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.673+0000 7ffbbf5ff700 1 -- 172.21.15.53:0/1160925260 wait complete. 2023-11-28T11:20:31.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:20:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.896+0000 7f01913ff700 1 Processor -- start 2023-11-28T11:20:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.896+0000 7f01913ff700 1 -- start start 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.897+0000 7f01913ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 0x7f018c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f01913ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018c0d1950 con 0x7f018c0cd7c0 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f01913ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018c0d24f0 con 0x7f018c161340 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f01913ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0b5800 con 0x7f018c15c9b0 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018b7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f018c0cd7c0 0x7f018c0b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38134/0 (socket says 172.21.15.53:38134) 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018b7fe700 1 -- 172.21.15.53:0/2513431741 learned_addr learned my addr 172.21.15.53:0/2513431741 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018affd700 1 --2- 172.21.15.53:0/2513431741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 0x7f018c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928268804 0 0) 0x7f018c0d1950 con 0x7f018c0cd7c0 2023-11-28T11:20:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01780035b0 con 0x7f018c0cd7c0 2023-11-28T11:20:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.898+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774137337 0 0) 0x7f018c0d24f0 con 0x7f018c161340 2023-11-28T11:20:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018c0d1950 con 0x7f018c161340 2023-11-28T11:20:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018affd700 1 -- 172.21.15.53:0/2513431741 >> v1:172.21.15.157:6789/0 conn(0x7f018c161340 legacy=0x7f018c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018affd700 1 -- 172.21.15.53:0/2513431741 >> v1:172.21.15.53:6789/0 conn(0x7f018c0cd7c0 legacy=0x7f018c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018affd700 1 -- 172.21.15.53:0/2513431741 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c0b5520 con 0x7f018c15c9b0 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018affd700 1 --2- 172.21.15.53:0/2513431741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 0x7f018c0b4bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc830c058d2f412 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c011830 con 0x7f018c15c9b0 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c011990 con 0x7f018c15c9b0 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.899+0000 7f018a7fc700 1 -- 172.21.15.53:0/2513431741 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c011d20 con 0x7f018c15c9b0 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 -- 172.21.15.53:0/2513431741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 msgr2=0x7f018c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 --2- 172.21.15.53:0/2513431741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 0x7f018c0b4bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c009b80 comp rx=0 tx=0).stop 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 -- 172.21.15.53:0/2513431741 shutdown_connections 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 --2- 172.21.15.53:0/2513431741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c15c9b0 0x7f018c0b4bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 -- 172.21.15.53:0/2513431741 >> 172.21.15.53:0/2513431741 conn(0x7f018c072600 msgr2=0x7f018c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 -- 172.21.15.53:0/2513431741 shutdown_connections 2023-11-28T11:20:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.900+0000 7f01913ff700 1 -- 172.21.15.53:0/2513431741 wait complete. 2023-11-28T11:20:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.902+0000 7f01913ff700 1 Processor -- start 2023-11-28T11:20:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.903+0000 7f01913ff700 1 -- start start 2023-11-28T11:20:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c0cd7c0 0x7f018c0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018c161340 0x7f018c1e7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0c9af0 con 0x7f018c15c9b0 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c157c00 con 0x7f018c161340 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f01913ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0c8f70 con 0x7f018c0cd7c0 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f0183fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f018affd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c0cd7c0 0x7f018c0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f0183fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44700/0 (socket says 172.21.15.53:44700) 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f0183fff700 1 -- 172.21.15.53:0/1922113402 learned_addr learned my addr 172.21.15.53:0/1922113402 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.904+0000 7f018b7fe700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018c161340 0x7f018c1e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 -- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c0cd7c0 msgr2=0x7f018c0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c0cd7c0 0x7f018c0c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 -- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018c161340 msgr2=0x7f018c1e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018c161340 0x7f018c1e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 -- 172.21.15.53:0/1922113402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0094a0 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f0183fff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f017400c530 tx=0x7f017400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400cea0 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01740052f0 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c0d15e0 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174003680 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.905+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c0d1300 con 0x7f018c15c9b0 2023-11-28T11:20:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.906+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c0b5130 con 0x7f018c15c9b0 2023-11-28T11:20:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.907+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0174019030 con 0x7f018c15c9b0 2023-11-28T11:20:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.908+0000 7f01837fe700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 0x7f016c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.908+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0174084ed0 con 0x7f018c15c9b0 2023-11-28T11:20:31.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.909+0000 7f018affd700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 0x7f016c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.910+0000 7f018affd700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 0x7f016c06a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f017c0097c0 tx=0x7f017c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:31.913+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0174051730 con 0x7f018c15c9b0 2023-11-28T11:20:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.116+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f018c0745c0 con 0x7f018c15c9b0 2023-11-28T11:20:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.116+0000 7f01837fe700 1 -- 172.21.15.53:0/1922113402 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01740170a0 con 0x7f018c15c9b0 2023-11-28T11:20:32.118 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:20:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 msgr2=0x7f016c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 0x7f016c06a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f017c0097c0 tx=0x7f017c01c040 comp rx=0 tx=0).stop 2023-11-28T11:20:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 msgr2=0x7f018c1576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f017400c530 tx=0x7f017400c840 comp rx=0 tx=0).stop 2023-11-28T11:20:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 shutdown_connections 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f016c0682f0 0x7f016c06a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f018c0cd7c0 0x7f018c0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018c161340 0x7f018c1e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 --2- 172.21.15.53:0/1922113402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f018c15c9b0 0x7f018c1576c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 >> 172.21.15.53:0/1922113402 conn(0x7f018c072600 msgr2=0x7f018c0b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 shutdown_connections 2023-11-28T11:20:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.119+0000 7f01913ff700 1 -- 172.21.15.53:0/1922113402 wait complete. 2023-11-28T11:20:32.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.338+0000 7f1c9f1ff700 1 Processor -- start 2023-11-28T11:20:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.338+0000 7f1c9f1ff700 1 -- start start 2023-11-28T11:20:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 0x7f1c980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 0x7f1c980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c980d2320 0x7f1c980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c980b5800 con 0x7f1c980d2320 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c980d1950 con 0x7f1c98161340 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.339+0000 7f1c9f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980b5520 con 0x7f1c9815c9b0 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c980d2320 0x7f1c980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c980d2320 0x7f1c980cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44716/0 (socket says 172.21.15.53:44716) 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 -- 172.21.15.53:0/1441032391 learned_addr learned my addr 172.21.15.53:0/1441032391 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c97fff700 1 --2- 172.21.15.53:0/1441032391 >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 0x7f1c980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9cf9b700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 0x7f1c980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 -- 172.21.15.53:0/1441032391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 msgr2=0x7f1c980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 0x7f1c980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 -- 172.21.15.53:0/1441032391 >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 msgr2=0x7f1c980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 --2- 172.21.15.53:0/1441032391 >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 0x7f1c980d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9d79c700 1 -- 172.21.15.53:0/1441032391 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980d1300 con 0x7f1c980d2320 2023-11-28T11:20:32.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c97fff700 1 --2- 172.21.15.53:0/1441032391 >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 0x7f1c980d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:32.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.340+0000 7f1c9cf9b700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 0x7f1c980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:32.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c9d79c700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c980d2320 0x7f1c980cd7c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1c9806bc00 tx=0x7f1c9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f2ae2e27488d85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c977fe700 1 -- 172.21.15.53:0/1441032391 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000e070 con 0x7f1c980d2320 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c977fe700 1 -- 172.21.15.53:0/1441032391 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000b9c0 con 0x7f1c980d2320 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c977fe700 1 -- 172.21.15.53:0/1441032391 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000bd70 con 0x7f1c980d2320 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/1441032391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c980d2320 msgr2=0x7f1c980cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.341+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c980d2320 0x7f1c980cd7c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1c9806bc00 tx=0x7f1c9000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/1441032391 shutdown_connections 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9815c9b0 0x7f1c980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/1441032391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c980d2320 0x7f1c980cd7c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/1441032391 >> v2:172.21.15.157:3300/0 conn(0x7f1c98161340 0x7f1c980d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/1441032391 >> 172.21.15.53:0/1441032391 conn(0x7f1c98072600 msgr2=0x7f1c980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/1441032391 shutdown_connections 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.342+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/1441032391 wait complete. 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.344+0000 7f1c9f1ff700 1 Processor -- start 2023-11-28T11:20:32.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.344+0000 7f1c9f1ff700 1 -- start start 2023-11-28T11:20:32.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.345+0000 7f1c9f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c980d2320 0x7f1c981542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.345+0000 7f1c9f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.345+0000 7f1c9f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c98161340 0x7f1c98151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c9f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980b6690 con 0x7f1c9815c9b0 2023-11-28T11:20:32.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c9f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981e8ae0 con 0x7f1c980d2320 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c9f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981e8800 con 0x7f1c98161340 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44718/0 (socket says 172.21.15.53:44718) 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 -- 172.21.15.53:0/276195785 learned_addr learned my addr 172.21.15.53:0/276195785 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c9cf9b700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c980d2320 0x7f1c981542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c9d79c700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c98161340 0x7f1c98151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 -- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c98161340 msgr2=0x7f1c98151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c98161340 0x7f1c98151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 -- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c980d2320 msgr2=0x7f1c981542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c980d2320 0x7f1c981542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.346+0000 7f1c97fff700 1 -- 172.21.15.53:0/276195785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90009b30 con 0x7f1c9815c9b0 2023-11-28T11:20:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c97fff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e810 tx=0x7f1c8400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84013610 con 0x7f1c9815c9b0 2023-11-28T11:20:32.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84013770 con 0x7f1c9815c9b0 2023-11-28T11:20:32.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84013b20 con 0x7f1c9815c9b0 2023-11-28T11:20:32.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c980b5130 con 0x7f1c9815c9b0 2023-11-28T11:20:32.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.347+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c980d15e0 con 0x7f1c9815c9b0 2023-11-28T11:20:32.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.348+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c980b52c0 con 0x7f1c9815c9b0 2023-11-28T11:20:32.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.349+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c84023070 con 0x7f1c9815c9b0 2023-11-28T11:20:32.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.350+0000 7f1c957fa700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 0x7f1c7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.350+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c84085830 con 0x7f1c9815c9b0 2023-11-28T11:20:32.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.350+0000 7f1c9cf9b700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 0x7f1c7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.351+0000 7f1c9cf9b700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 0x7f1c7c06a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c9815d220 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.354+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c84051fa0 con 0x7f1c9815c9b0 2023-11-28T11:20:32.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.606+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c980745c0 con 0x7f1c9815c9b0 2023-11-28T11:20:32.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.607+0000 7f1c957fa700 1 -- 172.21.15.53:0/276195785 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1c840555c0 con 0x7f1c9815c9b0 2023-11-28T11:20:32.608 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:32.608 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:20:32.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 msgr2=0x7f1c7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 0x7f1c7c06a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c9815d220 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 msgr2=0x7f1c98150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e810 tx=0x7f1c8400eb20 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 shutdown_connections 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1c7c0682f0 0x7f1c7c06a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c98161340 0x7f1c98151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c980d2320 0x7f1c981542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 --2- 172.21.15.53:0/276195785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c9815c9b0 0x7f1c98150220 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 >> 172.21.15.53:0/276195785 conn(0x7f1c98072600 msgr2=0x7f1c9815fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 shutdown_connections 2023-11-28T11:20:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.609+0000 7f1c9f1ff700 1 -- 172.21.15.53:0/276195785 wait complete. 2023-11-28T11:20:32.612 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:20:32.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.829+0000 7efcc39ff700 1 Processor -- start 2023-11-28T11:20:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.830+0000 7efcc39ff700 1 -- start start 2023-11-28T11:20:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc39ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efcbc15cd30 0x7efcbc161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc39ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcbc06a980 0x7efcbc0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc39ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0d0840 con 0x7efcbc0cfda0 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc39ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0d0b20 con 0x7efcbc06a980 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc39ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc160670 con 0x7efcbc15cd30 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc1f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcbc06a980 0x7efcbc0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc1f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcbc06a980 0x7efcbc0d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44722/0 (socket says 172.21.15.53:44722) 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc1f9c700 1 -- 172.21.15.53:0/1320123540 learned_addr learned my addr 172.21.15.53:0/1320123540 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.831+0000 7efcc179b700 1 --2- 172.21.15.53:0/1320123540 >> v2:172.21.15.157:3300/0 conn(0x7efcbc15cd30 0x7efcbc161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcc1f9c700 1 -- 172.21.15.53:0/1320123540 >> v2:172.21.15.157:3300/0 conn(0x7efcbc15cd30 msgr2=0x7efcbc161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcc1f9c700 1 --2- 172.21.15.53:0/1320123540 >> v2:172.21.15.157:3300/0 conn(0x7efcbc15cd30 0x7efcbc161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcc1f9c700 1 -- 172.21.15.53:0/1320123540 >> v1:172.21.15.157:6789/0 conn(0x7efcbc0cfda0 legacy=0x7efcbc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcc1f9c700 1 -- 172.21.15.53:0/1320123540 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc160950 con 0x7efcbc06a980 2023-11-28T11:20:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcc1f9c700 1 --2- 172.21.15.53:0/1320123540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc06a980 0x7efcbc0d0240 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009e50 tx=0x7efcb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9eb1bd5cd37fde1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcb3fff700 1 -- 172.21.15.53:0/1320123540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400e070 con 0x7efcbc06a980 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcb3fff700 1 -- 172.21.15.53:0/1320123540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400b980 con 0x7efcbc06a980 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.832+0000 7efcb3fff700 1 -- 172.21.15.53:0/1320123540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400bd30 con 0x7efcbc06a980 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 -- 172.21.15.53:0/1320123540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc06a980 msgr2=0x7efcbc0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 --2- 172.21.15.53:0/1320123540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc06a980 0x7efcbc0d0240 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009e50 tx=0x7efcb400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 -- 172.21.15.53:0/1320123540 shutdown_connections 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 --2- 172.21.15.53:0/1320123540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc06a980 0x7efcbc0d0240 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 --2- 172.21.15.53:0/1320123540 >> v2:172.21.15.157:3300/0 conn(0x7efcbc15cd30 0x7efcbc161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 -- 172.21.15.53:0/1320123540 >> 172.21.15.53:0/1320123540 conn(0x7efcbc06b380 msgr2=0x7efcbc06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 -- 172.21.15.53:0/1320123540 shutdown_connections 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.833+0000 7efcc39ff700 1 -- 172.21.15.53:0/1320123540 wait complete. 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.835+0000 7efcc39ff700 1 Processor -- start 2023-11-28T11:20:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.835+0000 7efcc39ff700 1 -- start start 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcbc06a980 0x7efcbc0c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcbc15cd30 0x7efcbc15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b42e0 con 0x7efcbc0cfda0 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b36d0 con 0x7efcbc06a980 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc39ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b39b0 con 0x7efcbc15cd30 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44730/0 (socket says 172.21.15.53:44730) 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 -- 172.21.15.53:0/2626146189 learned_addr learned my addr 172.21.15.53:0/2626146189 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc179b700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcbc06a980 0x7efcbc0c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc1f9c700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcbc15cd30 0x7efcbc15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 -- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcbc15cd30 msgr2=0x7efcbc15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcbc15cd30 0x7efcbc15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 -- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcbc06a980 msgr2=0x7efcbc0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcbc06a980 0x7efcbc0c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.836+0000 7efcc0f9a700 1 -- 172.21.15.53:0/2626146189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb4009b30 con 0x7efcbc0cfda0 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcc0f9a700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c490 tx=0x7efcac00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00cea0 con 0x7efcbc0cfda0 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac0052f0 con 0x7efcbc0cfda0 2023-11-28T11:20:32.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0cc280 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac006860 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.837+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc160c30 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.838+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc070de0 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.839+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efcac010050 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.840+0000 7efcb1ffb700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 0x7efca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.840+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efcac084de0 con 0x7efcbc0cfda0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.840+0000 7efcc179b700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 0x7efca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.841+0000 7efcc179b700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 0x7efca006a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006b20 tx=0x7efcb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:32.844+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efcac04a5e0 con 0x7efcbc0cfda0 2023-11-28T11:20:33.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.049+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcbc0745c0 con 0x7efcbc0cfda0 2023-11-28T11:20:33.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.050+0000 7efcb1ffb700 1 -- 172.21.15.53:0/2626146189 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17979 (secure 0 0 0) 0x7efcac05a020 con 0x7efcbc0cfda0 2023-11-28T11:20:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:33.054 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:45.403641+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 msgr2=0x7efca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 0x7efca006a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006b20 tx=0x7efcb8008040 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 msgr2=0x7efcbc0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c490 tx=0x7efcac00c7a0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 shutdown_connections 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7efca00682f0 0x7efca006a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcbc15cd30 0x7efcbc15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcbc06a980 0x7efcbc0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 --2- 172.21.15.53:0/2626146189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcbc0cfda0 0x7efcbc0b2cf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.052+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 >> 172.21.15.53:0/2626146189 conn(0x7efcbc06b380 msgr2=0x7efcbc0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.053+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 shutdown_connections 2023-11-28T11:20:33.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.053+0000 7efcc39ff700 1 -- 172.21.15.53:0/2626146189 wait complete. 2023-11-28T11:20:33.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:20:33.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:33.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.273+0000 7f1aa33ff700 1 Processor -- start 2023-11-28T11:20:33.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.273+0000 7f1aa33ff700 1 -- start start 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa33ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 0x7f1a9c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa33ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a9c0d2320 0x7f1a9c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa33ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0d1950 con 0x7f1a9c15c9b0 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa33ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b5800 con 0x7f1a9c0d2320 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa33ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b5300 con 0x7f1a9c161340 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa199c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a9c0d2320 0x7f1a9c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa119b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1a9c15c9b0 0x7f1a9c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38150/0 (socket says 172.21.15.53:38150) 2023-11-28T11:20:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.274+0000 7f1aa119b700 1 -- 172.21.15.53:0/1515258454 learned_addr learned my addr 172.21.15.53:0/1515258454 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa099a700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 0x7f1a9c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa199c700 1 -- 172.21.15.53:0/1515258454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 msgr2=0x7f1a9c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa199c700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 0x7f1a9c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1a93fff700 1 -- 172.21.15.53:0/1515258454 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310703070 0 0) 0x7f1a9c0d1950 con 0x7f1a9c15c9b0 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa199c700 1 -- 172.21.15.53:0/1515258454 >> v1:172.21.15.53:6789/0 conn(0x7f1a9c15c9b0 legacy=0x7f1a9c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa199c700 1 -- 172.21.15.53:0/1515258454 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0b5520 con 0x7f1a9c0d2320 2023-11-28T11:20:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa099a700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 0x7f1a9c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1aa199c700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0cd7c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009dc0 tx=0x7f1a9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6f223302a05882 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1a93fff700 1 -- 172.21.15.53:0/1515258454 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400e070 con 0x7f1a9c0d2320 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1a93fff700 1 -- 172.21.15.53:0/1515258454 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400bb90 con 0x7f1a9c0d2320 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.275+0000 7f1a93fff700 1 -- 172.21.15.53:0/1515258454 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94012430 con 0x7f1a9c0d2320 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 -- 172.21.15.53:0/1515258454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 msgr2=0x7f1a9c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0cd7c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009dc0 tx=0x7f1a9400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 -- 172.21.15.53:0/1515258454 shutdown_connections 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0cd7c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/1515258454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c161340 0x7f1a9c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 -- 172.21.15.53:0/1515258454 >> 172.21.15.53:0/1515258454 conn(0x7f1a9c072600 msgr2=0x7f1a9c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 -- 172.21.15.53:0/1515258454 shutdown_connections 2023-11-28T11:20:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.276+0000 7f1aa33ff700 1 -- 172.21.15.53:0/1515258454 wait complete. 2023-11-28T11:20:33.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.278+0000 7f1aa33ff700 1 Processor -- start 2023-11-28T11:20:33.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.278+0000 7f1aa33ff700 1 -- start start 2023-11-28T11:20:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c15c9b0 0x7f1a9c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 0x7f1a9c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c154f30 con 0x7f1a9c0d2320 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c152de0 con 0x7f1a9c161340 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa33ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c1530c0 con 0x7f1a9c15c9b0 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44740/0 (socket says 172.21.15.53:44740) 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 -- 172.21.15.53:0/3409801630 learned_addr learned my addr 172.21.15.53:0/3409801630 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa099a700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c15c9b0 0x7f1a9c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa199c700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 0x7f1a9c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 -- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c15c9b0 msgr2=0x7f1a9c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c15c9b0 0x7f1a9c1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 -- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 msgr2=0x7f1a9c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 0x7f1a9c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa119b700 1 -- 172.21.15.53:0/3409801630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009a70 con 0x7f1a9c0d2320 2023-11-28T11:20:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.279+0000 7f1aa199c700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 0x7f1a9c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1aa119b700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800c550 tx=0x7f1a9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98004c80 con 0x7f1a9c0d2320 2023-11-28T11:20:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a98004e00 con 0x7f1a9c0d2320 2023-11-28T11:20:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007310 con 0x7f1a9c0d2320 2023-11-28T11:20:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c0d15e0 con 0x7f1a9c0d2320 2023-11-28T11:20:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.280+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c0d1300 con 0x7f1a9c0d2320 2023-11-28T11:20:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.281+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c15cd80 con 0x7f1a9c0d2320 2023-11-28T11:20:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.282+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a9801a020 con 0x7f1a9c0d2320 2023-11-28T11:20:33.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.283+0000 7f1a91ffb700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 0x7f1a8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.283+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a98085930 con 0x7f1a9c0d2320 2023-11-28T11:20:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.284+0000 7f1aa099a700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 0x7f1a8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.284+0000 7f1aa099a700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 0x7f1a8006a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006fd0 tx=0x7f1a8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.287+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a980520a0 con 0x7f1a9c0d2320 2023-11-28T11:20:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.534+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a9c0745c0 con 0x7f1a9c0d2320 2023-11-28T11:20:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.535+0000 7f1a91ffb700 1 -- 172.21.15.53:0/3409801630 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1a980556c0 con 0x7f1a9c0d2320 2023-11-28T11:20:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:33.536 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 msgr2=0x7f1a8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 0x7f1a8006a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006fd0 tx=0x7f1a8c008040 comp rx=0 tx=0).stop 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 msgr2=0x7f1a9c0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800c550 tx=0x7f1a9800c910 comp rx=0 tx=0).stop 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 shutdown_connections 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1a800682f0 0x7f1a8006a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a9c161340 0x7f1a9c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c0d2320 0x7f1a9c0c6060 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.537+0000 7f1aa33ff700 1 --2- 172.21.15.53:0/3409801630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a9c15c9b0 0x7f1a9c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.538+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 >> 172.21.15.53:0/3409801630 conn(0x7f1a9c072600 msgr2=0x7f1a9c06b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.538+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 shutdown_connections 2023-11-28T11:20:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.538+0000 7f1aa33ff700 1 -- 172.21.15.53:0/3409801630 wait complete. 2023-11-28T11:20:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:20:33.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:33.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.757+0000 7fc29fdff700 1 Processor -- start 2023-11-28T11:20:33.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.757+0000 7fc29fdff700 1 -- start start 2023-11-28T11:20:33.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.758+0000 7fc29fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29815c9b0 0x7fc2980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.758+0000 7fc29fdff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2980cd7c0 0x7fc2980cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.758+0000 7fc29fdff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2980ce160 con 0x7fc298161340 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.758+0000 7fc29fdff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2980b5800 con 0x7fc2980cd7c0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.758+0000 7fc29fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980d1950 con 0x7fc29815c9b0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2980cd7c0 0x7fc2980cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2980cd7c0 0x7fc2980cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44754/0 (socket says 172.21.15.53:44754) 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 -- 172.21.15.53:0/1993493184 learned_addr learned my addr 172.21.15.53:0/1993493184 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29db9b700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29815c9b0 0x7fc2980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 -- 172.21.15.53:0/1993493184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29815c9b0 msgr2=0x7fc2980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29815c9b0 0x7fc2980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 -- 172.21.15.53:0/1993493184 >> v1:172.21.15.157:6789/0 conn(0x7fc298161340 legacy=0x7fc2980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 -- 172.21.15.53:0/1993493184 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980d24f0 con 0x7fc2980cd7c0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.759+0000 7fc29e39c700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2980cd7c0 0x7fc2980cdc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc290009e50 tx=0x7fc29000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4055d7a1862941c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29cb99700 1 -- 172.21.15.53:0/1993493184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000e070 con 0x7fc2980cd7c0 2023-11-28T11:20:33.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29cb99700 1 -- 172.21.15.53:0/1993493184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000b9c0 con 0x7fc2980cd7c0 2023-11-28T11:20:33.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29cb99700 1 -- 172.21.15.53:0/1993493184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000bd70 con 0x7fc2980cd7c0 2023-11-28T11:20:33.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 -- 172.21.15.53:0/1993493184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2980cd7c0 msgr2=0x7fc2980cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2980cd7c0 0x7fc2980cdc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc290009e50 tx=0x7fc29000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 -- 172.21.15.53:0/1993493184 shutdown_connections 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2980cd7c0 0x7fc2980cdc20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 --2- 172.21.15.53:0/1993493184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29815c9b0 0x7fc2980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 -- 172.21.15.53:0/1993493184 >> 172.21.15.53:0/1993493184 conn(0x7fc298072600 msgr2=0x7fc2980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 -- 172.21.15.53:0/1993493184 shutdown_connections 2023-11-28T11:20:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.760+0000 7fc29fdff700 1 -- 172.21.15.53:0/1993493184 wait complete. 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.762+0000 7fc29fdff700 1 Processor -- start 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.762+0000 7fc29fdff700 1 -- start start 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 0x7fc29814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29815c9b0 0x7fc2980b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980c9530 con 0x7fc298161340 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2981f2ca0 con 0x7fc29815c9b0 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980ca3e0 con 0x7fc2980cd7c0 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29e39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29e39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44770/0 (socket says 172.21.15.53:44770) 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29e39c700 1 -- 172.21.15.53:0/597567638 learned_addr learned my addr 172.21.15.53:0/597567638 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29d39a700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29815c9b0 0x7fc2980b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.763+0000 7fc29db9b700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 0x7fc29814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 -- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 msgr2=0x7fc29814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 0x7fc29814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 -- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29815c9b0 msgr2=0x7fc2980b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29815c9b0 0x7fc2980b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 -- 172.21.15.53:0/597567638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290009b30 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29db9b700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 0x7fc29814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29e39c700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b730 tx=0x7fc290004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2900048e0 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2900099b0 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2980d1300 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.764+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29001b670 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.765+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2980b5520 con 0x7fc298161340 2023-11-28T11:20:33.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.765+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2980d15e0 con 0x7fc298161340 2023-11-28T11:20:33.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.766+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc29002b070 con 0x7fc298161340 2023-11-28T11:20:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.767+0000 7fc28e7fc700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.767+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc290087500 con 0x7fc298161340 2023-11-28T11:20:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.768+0000 7fc29db9b700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.768+0000 7fc29db9b700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 0x7fc27c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc294007a50 tx=0x7fc294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:33.771+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc290053c70 con 0x7fc298161340 2023-11-28T11:20:34.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.022+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2980cea60 con 0x7fc298161340 2023-11-28T11:20:34.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.023+0000 7fc28e7fc700 1 -- 172.21.15.53:0/597567638 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fc290057290 con 0x7fc298161340 2023-11-28T11:20:34.024 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:34.024 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:19:44.401098+0000","modified":"2023-11-28T11:19:47.410724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55007":{"gid":55007,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4065027310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4065027310},{"type":"v1","addr":"172.21.15.53:6835","nonce":4065027310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 msgr2=0x7fc27c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 0x7fc27c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc294007a50 tx=0x7fc294008040 comp rx=0 tx=0).stop 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 msgr2=0x7fc2980b88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b730 tx=0x7fc290004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 shutdown_connections 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2980cd7c0 0x7fc29814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc298161340 0x7fc2980b88e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 --2- 172.21.15.53:0/597567638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29815c9b0 0x7fc2980b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 >> 172.21.15.53:0/597567638 conn(0x7fc298072600 msgr2=0x7fc2980b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 shutdown_connections 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.025+0000 7fc29fdff700 1 -- 172.21.15.53:0/597567638 wait complete. 2023-11-28T11:20:34.027 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 24 2023-11-28T11:20:34.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.245+0000 7f57d95ff700 1 Processor -- start 2023-11-28T11:20:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.245+0000 7f57d95ff700 1 -- start start 2023-11-28T11:20:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57d95ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57d4161340 0x7f57d40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57d95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57d95ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40d1950 con 0x7f57d415c9b0 2023-11-28T11:20:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57d95ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d40b5800 con 0x7f57d4161340 2023-11-28T11:20:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57d95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b5300 con 0x7f57d40d2320 2023-11-28T11:20:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57cbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57d4161340 0x7f57d40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57cbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57d4161340 0x7f57d40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44782/0 (socket says 172.21.15.53:44782) 2023-11-28T11:20:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.246+0000 7f57cbfff700 1 -- 172.21.15.53:0/1405525413 learned_addr learned my addr 172.21.15.53:0/1405525413 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57d37fe700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57cbfff700 1 -- 172.21.15.53:0/1405525413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 msgr2=0x7f57d40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57cbfff700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57cbfff700 1 -- 172.21.15.53:0/1405525413 >> v1:172.21.15.53:6789/0 conn(0x7f57d415c9b0 legacy=0x7f57d40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57cbfff700 1 -- 172.21.15.53:0/1405525413 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40b5520 con 0x7f57d4161340 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57d37fe700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57cbfff700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4161340 0x7f57d40d1de0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0098f0 tx=0x7f57bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808e5b3a01d75c2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57d27fc700 1 -- 172.21.15.53:0/1405525413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00b8a0 con 0x7f57d4161340 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57d27fc700 1 -- 172.21.15.53:0/1405525413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00ba00 con 0x7f57d4161340 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.247+0000 7f57d27fc700 1 -- 172.21.15.53:0/1405525413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00bd90 con 0x7f57d4161340 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 -- 172.21.15.53:0/1405525413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4161340 msgr2=0x7f57d40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4161340 0x7f57d40d1de0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0098f0 tx=0x7f57bc009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 -- 172.21.15.53:0/1405525413 shutdown_connections 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 --2- 172.21.15.53:0/1405525413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4161340 0x7f57d40d1de0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 -- 172.21.15.53:0/1405525413 >> 172.21.15.53:0/1405525413 conn(0x7f57d4072600 msgr2=0x7f57d40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 -- 172.21.15.53:0/1405525413 shutdown_connections 2023-11-28T11:20:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.248+0000 7f57d95ff700 1 -- 172.21.15.53:0/1405525413 wait complete. 2023-11-28T11:20:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.250+0000 7f57d95ff700 1 Processor -- start 2023-11-28T11:20:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.250+0000 7f57d95ff700 1 -- start start 2023-11-28T11:20:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 0x7f57d41e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40ce1e0 con 0x7f57d415c9b0 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d415e030 con 0x7f57d4161340 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d415da70 con 0x7f57d40d2320 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44794/0 (socket says 172.21.15.53:44794) 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 -- 172.21.15.53:0/4267305400 learned_addr learned my addr 172.21.15.53:0/4267305400 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d2ffd700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 -- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 msgr2=0x7f57d40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d37fe700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 0x7f57d41e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 -- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 msgr2=0x7f57d41e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 0x7f57d41e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57cbfff700 1 -- 172.21.15.53:0/4267305400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009550 con 0x7f57d415c9b0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.251+0000 7f57d2ffd700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57d37fe700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 0x7f57d41e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57cbfff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000c00 tx=0x7f57bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004570 con 0x7f57d415c9b0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc01f070 con 0x7f57d415c9b0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d40d15e0 con 0x7f57d415c9b0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0038e0 con 0x7f57d415c9b0 2023-11-28T11:20:34.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.252+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d40d1300 con 0x7f57d415c9b0 2023-11-28T11:20:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.253+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d415cd80 con 0x7f57d415c9b0 2023-11-28T11:20:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.254+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57bc02a020 con 0x7f57d415c9b0 2023-11-28T11:20:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.255+0000 7f57cb7fe700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 0x7f57b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.255+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57bc087470 con 0x7f57d415c9b0 2023-11-28T11:20:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.255+0000 7f57d2ffd700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 0x7f57b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.256+0000 7f57d2ffd700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 0x7f57b406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57d415eac0 tx=0x7f57c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.259+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57bc053be0 con 0x7f57d415c9b0 2023-11-28T11:20:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.467+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d40745c0 con 0x7f57d415c9b0 2023-11-28T11:20:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.469+0000 7f57cb7fe700 1 -- 172.21.15.53:0/4267305400 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f57bc057200 con 0x7f57d415c9b0 2023-11-28T11:20:34.473 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:34.473 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:19:45.403641+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:19:42.218381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:19:43.389004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 msgr2=0x7f57b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 0x7f57b406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57d415eac0 tx=0x7f57c4009040 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 msgr2=0x7f57d415d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000c00 tx=0x7f57bc004750 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 shutdown_connections 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57b40682f0 0x7f57b406a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d40d2320 0x7f57d40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d4161340 0x7f57d41e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 --2- 172.21.15.53:0/4267305400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d415c9b0 0x7f57d415d0b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 >> 172.21.15.53:0/4267305400 conn(0x7f57d4072600 msgr2=0x7f57d40b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 shutdown_connections 2023-11-28T11:20:34.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.471+0000 7f57d95ff700 1 -- 172.21.15.53:0/4267305400 wait complete. 2023-11-28T11:20:34.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:20:34.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.691+0000 7f15243ff700 1 Processor -- start 2023-11-28T11:20:34.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.691+0000 7f15243ff700 1 -- start start 2023-11-28T11:20:34.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f15243ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 0x7f151c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f15243ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 0x7f151c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f15243ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c0d1950 con 0x7f151c15c9b0 2023-11-28T11:20:34.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f15243ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0b5800 con 0x7f151c0d2320 2023-11-28T11:20:34.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f15243ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b5300 con 0x7f151c161340 2023-11-28T11:20:34.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f152199a700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 0x7f151c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f152299c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 0x7f151c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f152219b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f151c15c9b0 0x7f151c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:39898/0 (socket says 172.21.15.53:39898) 2023-11-28T11:20:34.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.693+0000 7f152219b700 1 -- 172.21.15.53:0/163730087 learned_addr learned my addr 172.21.15.53:0/163730087 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f1521199700 1 -- 172.21.15.53:0/163730087 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145408011 0 0) 0x7f151c0d1950 con 0x7f151c15c9b0 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152199a700 1 -- 172.21.15.53:0/163730087 >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 msgr2=0x7f151c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152199a700 1 --2- 172.21.15.53:0/163730087 >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 0x7f151c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152199a700 1 -- 172.21.15.53:0/163730087 >> v1:172.21.15.157:6789/0 conn(0x7f151c15c9b0 legacy=0x7f151c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152199a700 1 -- 172.21.15.53:0/163730087 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c0b5520 con 0x7f151c161340 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152299c700 1 --2- 172.21.15.53:0/163730087 >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 0x7f151c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.694+0000 7f152199a700 1 --2- 172.21.15.53:0/163730087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 0x7f151c0d1de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f150c009840 tx=0x7f150c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b8780a76c37f46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f1521199700 1 -- 172.21.15.53:0/163730087 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c011830 con 0x7f151c161340 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f1521199700 1 -- 172.21.15.53:0/163730087 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c011990 con 0x7f151c161340 2023-11-28T11:20:34.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f1521199700 1 -- 172.21.15.53:0/163730087 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c011d20 con 0x7f151c161340 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 -- 172.21.15.53:0/163730087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 msgr2=0x7f151c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 --2- 172.21.15.53:0/163730087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 0x7f151c0d1de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f150c009840 tx=0x7f150c009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 -- 172.21.15.53:0/163730087 shutdown_connections 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 --2- 172.21.15.53:0/163730087 >> v2:172.21.15.157:3300/0 conn(0x7f151c0d2320 0x7f151c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 --2- 172.21.15.53:0/163730087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c161340 0x7f151c0d1de0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 -- 172.21.15.53:0/163730087 >> 172.21.15.53:0/163730087 conn(0x7f151c072600 msgr2=0x7f151c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 -- 172.21.15.53:0/163730087 shutdown_connections 2023-11-28T11:20:34.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.695+0000 7f15243ff700 1 -- 172.21.15.53:0/163730087 wait complete. 2023-11-28T11:20:34.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.697+0000 7f15243ff700 1 Processor -- start 2023-11-28T11:20:34.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.697+0000 7f15243ff700 1 -- start start 2023-11-28T11:20:34.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c15c9b0 0x7f151c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151c161340 0x7f151c1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c8540 con 0x7f151c0d2320 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c87e0 con 0x7f151c161340 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f15243ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c8ac0 con 0x7f151c15c9b0 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f152219b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f152219b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44798/0 (socket says 172.21.15.53:44798) 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.698+0000 7f152219b700 1 -- 172.21.15.53:0/682441610 learned_addr learned my addr 172.21.15.53:0/682441610 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152199a700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c15c9b0 0x7f151c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152299c700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151c161340 0x7f151c1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 -- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c15c9b0 msgr2=0x7f151c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c15c9b0 0x7f151c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 -- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151c161340 msgr2=0x7f151c1f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151c161340 0x7f151c1f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 -- 172.21.15.53:0/682441610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009a70 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f152219b700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f15180011e0 tx=0x7f151800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.699+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518004c80 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.700+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518004de0 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.700+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c0094a0 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.700+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518007310 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.700+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c0d15e0 con 0x7f151c0d2320 2023-11-28T11:20:34.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.701+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0d1300 con 0x7f151c0d2320 2023-11-28T11:20:34.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.702+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f151801a020 con 0x7f151c0d2320 2023-11-28T11:20:34.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.702+0000 7f1512ffd700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 0x7f150006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:34.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.702+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1518085760 con 0x7f151c0d2320 2023-11-28T11:20:34.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.703+0000 7f152199a700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 0x7f150006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:34.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.704+0000 7f152199a700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 0x7f150006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f150c0090b0 tx=0x7f150c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:34.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.707+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1518051fc0 con 0x7f151c0d2320 2023-11-28T11:20:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:34.955+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f151c15cd80 con 0x7f151c0d2320 2023-11-28T11:20:35.877 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-11-28T11:20:35.876+0000 7fa6ddbf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:20:35.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.884+0000 7f1512ffd700 1 -- 172.21.15.53:0/682441610 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f15180555e0 con 0x7f151c0d2320 2023-11-28T11:20:35.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 msgr2=0x7f150006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:35.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 0x7f150006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f150c0090b0 tx=0x7f150c01c040 comp rx=0 tx=0).stop 2023-11-28T11:20:35.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 msgr2=0x7f151c0c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f15180011e0 tx=0x7f151800ca30 comp rx=0 tx=0).stop 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 shutdown_connections 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f15000682f0 0x7f150006a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151c161340 0x7f151c1f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f151c0d2320 0x7f151c0c61f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 --2- 172.21.15.53:0/682441610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f151c15c9b0 0x7f151c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 >> 172.21.15.53:0/682441610 conn(0x7f151c072600 msgr2=0x7f151c0b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 shutdown_connections 2023-11-28T11:20:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:35.888+0000 7f15243ff700 1 -- 172.21.15.53:0/682441610 wait complete. 2023-11-28T11:20:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:20:35.906 INFO:tasks.ceph.mds.c.smithi053.stderr: -6602> 2023-11-28T11:19:48.870+0000 7f764b1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:20:35.917 INFO:tasks.ceph.mds.c.smithi053.stderr: -3043> 2023-11-28T11:19:54.496+0000 7f76479ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={55112=0-4194304@1} caps={55112=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=55112 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5604cf9b0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:20:35.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:20:35.969 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:20:36.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.133+0000 7fc27edff700 1 Processor -- start 2023-11-28T11:20:36.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.133+0000 7fc27edff700 1 -- start start 2023-11-28T11:20:36.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27edff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2780d2320 0x7fc2780cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27edff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2780d1950 con 0x7fc27815c9b0 2023-11-28T11:20:36.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27edff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2780b5800 con 0x7fc2780d2320 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780b5300 con 0x7fc278161340 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.134+0000 7fc27d39c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2780d2320 0x7fc2780cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2780d2320 0x7fc2780cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44822/0 (socket says 172.21.15.53:44822) 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 -- 172.21.15.53:0/3962238430 learned_addr learned my addr 172.21.15.53:0/3962238430 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc277fff700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 -- 172.21.15.53:0/3962238430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 msgr2=0x7fc2780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2780d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 -- 172.21.15.53:0/3962238430 >> v1:172.21.15.157:6789/0 conn(0x7fc27815c9b0 legacy=0x7fc2780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 -- 172.21.15.53:0/3962238430 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2780b5520 con 0x7fc2780d2320 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc27d39c700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc2780cd7c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc270009e80 tx=0x7fc27000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8406068abf191680 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.135+0000 7fc2777fe700 1 -- 172.21.15.53:0/3962238430 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000e070 con 0x7fc2780d2320 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc2777fe700 1 -- 172.21.15.53:0/3962238430 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27000b8d0 con 0x7fc2780d2320 2023-11-28T11:20:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc2777fe700 1 -- 172.21.15.53:0/3962238430 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000bc80 con 0x7fc2780d2320 2023-11-28T11:20:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 -- 172.21.15.53:0/3962238430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 msgr2=0x7fc2780cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc2780cd7c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc270009e80 tx=0x7fc27000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 -- 172.21.15.53:0/3962238430 shutdown_connections 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc2780cd7c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 --2- 172.21.15.53:0/3962238430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2780d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 -- 172.21.15.53:0/3962238430 >> 172.21.15.53:0/3962238430 conn(0x7fc278072600 msgr2=0x7fc2780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 -- 172.21.15.53:0/3962238430 shutdown_connections 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.136+0000 7fc27edff700 1 -- 172.21.15.53:0/3962238430 wait complete. 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.138+0000 7fc27edff700 1 Processor -- start 2023-11-28T11:20:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.138+0000 7fc27edff700 1 -- start start 2023-11-28T11:20:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 0x7fc27815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2781e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781f3b90 con 0x7fc2780d2320 2023-11-28T11:20:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278157940 con 0x7fc27815c9b0 2023-11-28T11:20:36.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781f2f10 con 0x7fc278161340 2023-11-28T11:20:36.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27cb9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:36.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27cb9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44828/0 (socket says 172.21.15.53:44828) 2023-11-28T11:20:36.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.139+0000 7fc27cb9b700 1 -- 172.21.15.53:0/910549082 learned_addr learned my addr 172.21.15.53:0/910549082 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 -- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 msgr2=0x7fc2781e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2781e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc277fff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 0x7fc27815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 -- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 msgr2=0x7fc27815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 0x7fc27815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 -- 172.21.15.53:0/910549082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009b30 con 0x7fc2780d2320 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc277fff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 0x7fc27815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27cb9b700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c450 tx=0x7fc26c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0052d0 con 0x7fc2780d2320 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c005430 con 0x7fc2780d2320 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2780d15e0 con 0x7fc2780d2320 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.140+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0038c0 con 0x7fc2780d2320 2023-11-28T11:20:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.141+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2780d1300 con 0x7fc2780d2320 2023-11-28T11:20:36.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.142+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27815cd80 con 0x7fc2780d2320 2023-11-28T11:20:36.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.142+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc26c019030 con 0x7fc2780d2320 2023-11-28T11:20:36.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.144+0000 7fc2757fa700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 0x7fc25c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:36.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.144+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc26c085090 con 0x7fc2780d2320 2023-11-28T11:20:36.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.144+0000 7fc277fff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 0x7fc25c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:36.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.145+0000 7fc277fff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 0x7fc25c06a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc27815f770 tx=0x7fc264008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:36.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.147+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc26c0517a0 con 0x7fc2780d2320 2023-11-28T11:20:36.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.396+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc2780745c0 con 0x7fc2780d2320 2023-11-28T11:20:36.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.962+0000 7fc2757fa700 1 -- 172.21.15.53:0/910549082 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc26c054dc0 con 0x7fc2780d2320 2023-11-28T11:20:36.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.965+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 msgr2=0x7fc25c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.965+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 0x7fc25c06a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc27815f770 tx=0x7fc264008040 comp rx=0 tx=0).stop 2023-11-28T11:20:36.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.965+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 msgr2=0x7fc278157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:36.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.965+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c450 tx=0x7fc26c00c810 comp rx=0 tx=0).stop 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 shutdown_connections 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc25c0682f0 0x7fc25c06a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc278161340 0x7fc2781e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2780d2320 0x7fc278157400 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 --2- 172.21.15.53:0/910549082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27815c9b0 0x7fc27815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 >> 172.21.15.53:0/910549082 conn(0x7fc278072600 msgr2=0x7fc278160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 shutdown_connections 2023-11-28T11:20:36.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:36.966+0000 7fc27edff700 1 -- 172.21.15.53:0/910549082 wait complete. 2023-11-28T11:20:36.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:20:37.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.194+0000 7f00065ff700 1 Processor -- start 2023-11-28T11:20:37.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.194+0000 7f00065ff700 1 -- start start 2023-11-28T11:20:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.195+0000 7f00065ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00000cded0 0x7f0000161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.195+0000 7f00065ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.195+0000 7f00065ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0000160d70 con 0x7f000015a760 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.195+0000 7f00065ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000015e0d0 con 0x7f00000cded0 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.195+0000 7f00065ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000015ab30 con 0x7f000015dc30 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00000cded0 0x7f0000161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7f0004b9c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7efffffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f000015a760 0x7f00000cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:39916/0 (socket says 172.21.15.53:39916) 2023-11-28T11:20:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7efffffff700 1 -- 172.21.15.53:0/3481704954 learned_addr learned my addr 172.21.15.53:0/3481704954 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 -- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 msgr2=0x7f00000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 --2- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 -- 172.21.15.53:0/3481704954 >> v1:172.21.15.157:6789/0 conn(0x7f000015a760 legacy=0x7f00000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 -- 172.21.15.53:0/3481704954 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000b5130 con 0x7f00000cded0 2023-11-28T11:20:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.196+0000 7effff7fe700 1 --2- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f0000161200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a463addd06cb0eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7efffeffd700 1 -- 172.21.15.53:0/3481704954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011830 con 0x7f00000cded0 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7efffeffd700 1 -- 172.21.15.53:0/3481704954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4011990 con 0x7f00000cded0 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7efffeffd700 1 -- 172.21.15.53:0/3481704954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011d20 con 0x7f00000cded0 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 -- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 msgr2=0x7f0000161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 --2- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f0000161200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff4009b80 comp rx=0 tx=0).stop 2023-11-28T11:20:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 -- 172.21.15.53:0/3481704954 shutdown_connections 2023-11-28T11:20:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 --2- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 --2- 172.21.15.53:0/3481704954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f0000161200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 -- 172.21.15.53:0/3481704954 >> 172.21.15.53:0/3481704954 conn(0x7f0000072620 msgr2=0x7f00000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.197+0000 7f00065ff700 1 -- 172.21.15.53:0/3481704954 shutdown_connections 2023-11-28T11:20:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.198+0000 7f00065ff700 1 -- 172.21.15.53:0/3481704954 wait complete. 2023-11-28T11:20:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.199+0000 7f00065ff700 1 Processor -- start 2023-11-28T11:20:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.199+0000 7f00065ff700 1 -- start start 2023-11-28T11:20:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f000014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000014fc10 con 0x7f000015a760 2023-11-28T11:20:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00000ca240 con 0x7f00000cded0 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7f00065ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00000cb0a0 con 0x7f000015dc30 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.200+0000 7effff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44838/0 (socket says 172.21.15.53:44838) 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 -- 172.21.15.53:0/4293163680 learned_addr learned my addr 172.21.15.53:0/4293163680 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7efffffff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f000014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7f0004b9c700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 -- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 msgr2=0x7f00000cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 -- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 msgr2=0x7f000014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f000014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 -- 172.21.15.53:0/4293163680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40094a0 con 0x7f000015a760 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7effff7fe700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efff40090b0 tx=0x7efff4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.201+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40036a0 con 0x7f000015a760 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.202+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff401f070 con 0x7f000015a760 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.202+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00000b5670 con 0x7f000015a760 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.202+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4003a20 con 0x7f000015a760 2023-11-28T11:20:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.202+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000b5390 con 0x7f000015a760 2023-11-28T11:20:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.203+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0000160720 con 0x7f000015a760 2023-11-28T11:20:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.204+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efff400f050 con 0x7f000015a760 2023-11-28T11:20:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.204+0000 7efffcff9700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 0x7effec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.204+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6206+0+0 (secure 0 0 0) 0x7efff4021070 con 0x7f000015a760 2023-11-28T11:20:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.205+0000 7efffffff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 0x7effec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.205+0000 7efffffff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 0x7effec06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efff0005910 tx=0x7efff0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.209+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efff4053d40 con 0x7f000015a760 2023-11-28T11:20:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.415+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f000006ba10 con 0x7f000015a760 2023-11-28T11:20:37.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.892+0000 7efffcff9700 1 -- 172.21.15.53:0/4293163680 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7efff40150d0 con 0x7f000015a760 2023-11-28T11:20:37.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 msgr2=0x7effec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 0x7effec06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efff0005910 tx=0x7efff0009040 comp rx=0 tx=0).stop 2023-11-28T11:20:37.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 msgr2=0x7f00000c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:37.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efff40090b0 tx=0x7efff4004750 comp rx=0 tx=0).stop 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 shutdown_connections 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7effec0682f0 0x7effec06a7a0 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efff0005910 tx=0x7efff0009040 comp rx=0 tx=0).stop 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000015dc30 0x7f00000cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00000cded0 0x7f000014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 --2- 172.21.15.53:0/4293163680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f000015a760 0x7f00000c8dc0 secure :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efff40090b0 tx=0x7efff4004750 comp rx=0 tx=0).stop 2023-11-28T11:20:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.896+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 >> 172.21.15.53:0/4293163680 conn(0x7f0000072620 msgr2=0x7f00000d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.898+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 shutdown_connections 2023-11-28T11:20:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:37.898+0000 7f00065ff700 1 -- 172.21.15.53:0/4293163680 wait complete. 2023-11-28T11:20:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:20:37.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:20:38.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.130+0000 7fc64a1ff700 1 Processor -- start 2023-11-28T11:20:38.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.130+0000 7fc64a1ff700 1 -- start start 2023-11-28T11:20:38.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.131+0000 7fc64a1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc64415cd30 0x7fc644161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.131+0000 7fc64a1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6440cfda0 0x7fc6440d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.131+0000 7fc64a1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6440d0840 con 0x7fc64406a270 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.131+0000 7fc64a1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6440d0b20 con 0x7fc64415cd30 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.131+0000 7fc64a1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc644160670 con 0x7fc6440cfda0 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc64415cd30 0x7fc644161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc643fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc64406a270 0x7fc644070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38174/0 (socket says 172.21.15.53:38174) 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc643fff700 1 -- 172.21.15.53:0/3856099189 learned_addr learned my addr 172.21.15.53:0/3856099189 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc642ffd700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6440cfda0 0x7fc6440d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 -- 172.21.15.53:0/3856099189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6440cfda0 msgr2=0x7fc6440d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:38.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6440cfda0 0x7fc6440d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 -- 172.21.15.53:0/3856099189 >> v1:172.21.15.53:6789/0 conn(0x7fc64406a270 legacy=0x7fc644070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 -- 172.21.15.53:0/3856099189 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644160950 con 0x7fc64415cd30 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6427fc700 1 -- 172.21.15.53:0/3856099189 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105490968 0 0) 0x7fc6440d0840 con 0x7fc64406a270 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.132+0000 7fc6437fe700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc64415cd30 0x7fc644161150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc634009840 tx=0x7fc634009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1566adacba8e129d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc6427fc700 1 -- 172.21.15.53:0/3856099189 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011830 con 0x7fc64415cd30 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc6427fc700 1 -- 172.21.15.53:0/3856099189 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634011990 con 0x7fc64415cd30 2023-11-28T11:20:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc6427fc700 1 -- 172.21.15.53:0/3856099189 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011d20 con 0x7fc64415cd30 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 -- 172.21.15.53:0/3856099189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc64415cd30 msgr2=0x7fc644161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc64415cd30 0x7fc644161150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc634009840 tx=0x7fc634009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 -- 172.21.15.53:0/3856099189 shutdown_connections 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc64415cd30 0x7fc644161150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/3856099189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6440cfda0 0x7fc6440d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.133+0000 7fc64a1ff700 1 -- 172.21.15.53:0/3856099189 >> 172.21.15.53:0/3856099189 conn(0x7fc64406b380 msgr2=0x7fc64406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.134+0000 7fc64a1ff700 1 -- 172.21.15.53:0/3856099189 shutdown_connections 2023-11-28T11:20:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.134+0000 7fc64a1ff700 1 -- 172.21.15.53:0/3856099189 wait complete. 2023-11-28T11:20:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.136+0000 7fc64a1ff700 1 Processor -- start 2023-11-28T11:20:38.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.137+0000 7fc64a1ff700 1 -- start start 2023-11-28T11:20:38.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.138+0000 7fc64a1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc64a1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 0x7fc6441e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc64a1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 0x7fc6441e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc64a1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441e76c0 con 0x7fc6440cfda0 2023-11-28T11:20:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc64a1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441e73e0 con 0x7fc64415cd30 2023-11-28T11:20:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc64a1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440b38a0 con 0x7fc64406a270 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc642ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 0x7fc6441e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc643fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 0x7fc6441e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:59626/0 (socket says 172.21.15.53:59626) 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc643fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 0x7fc6441e8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:33972/0 (socket says 172.21.15.53:33972) 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc642ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 0x7fc6441e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44852/0 (socket says 172.21.15.53:44852) 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.139+0000 7fc6437fe700 1 -- 172.21.15.53:0/210303161 learned_addr learned my addr 172.21.15.53:0/210303161 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 -- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 msgr2=0x7fc6441e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 0x7fc6441e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 -- 172.21.15.53:0/210303161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 msgr2=0x7fc6441e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 0x7fc6441e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 -- 172.21.15.53:0/210303161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340094a0 con 0x7fc64406a270 2023-11-28T11:20:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.140+0000 7fc6437fe700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b90 tx=0x7fc634004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.141+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340036a0 con 0x7fc64406a270 2023-11-28T11:20:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.141+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63401e070 con 0x7fc64406a270 2023-11-28T11:20:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.141+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634003a20 con 0x7fc64406a270 2023-11-28T11:20:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.141+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6440cc280 con 0x7fc64406a270 2023-11-28T11:20:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.141+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc644160c30 con 0x7fc64406a270 2023-11-28T11:20:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.142+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64406a6f0 con 0x7fc64406a270 2023-11-28T11:20:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.143+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc63400f050 con 0x7fc64406a270 2023-11-28T11:20:38.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.144+0000 7fc62bfff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 0x7fc63006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:38.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.144+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fc634086b00 con 0x7fc64406a270 2023-11-28T11:20:38.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.144+0000 7fc642ffd700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 0x7fc63006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:38.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.145+0000 7fc642ffd700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 0x7fc63006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6440c2e90 tx=0x7fc638009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:38.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.150+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc634088e60 con 0x7fc64406a270 2023-11-28T11:20:38.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.353+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6440b3700 con 0x7fc64406a270 2023-11-28T11:20:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.967+0000 7fc62bfff700 1 -- 172.21.15.53:0/210303161 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc6440b3700 con 0x7fc64406a270 2023-11-28T11:20:38.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 msgr2=0x7fc63006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 0x7fc63006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6440c2e90 tx=0x7fc638009210 comp rx=0 tx=0).stop 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 msgr2=0x7fc6441516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b90 tx=0x7fc634004750 comp rx=0 tx=0).stop 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 shutdown_connections 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc6300682f0 0x7fc63006a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64406a270 0x7fc6441516a0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b90 tx=0x7fc634004750 comp rx=0 tx=0).stop 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc64415cd30 0x7fc6441e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 --2- 172.21.15.53:0/210303161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6440cfda0 0x7fc6441e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.972+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 >> 172.21.15.53:0/210303161 conn(0x7fc64406b380 msgr2=0x7fc6440cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.973+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 shutdown_connections 2023-11-28T11:20:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:38.973+0000 7fc64a1ff700 1 -- 172.21.15.53:0/210303161 wait complete. 2023-11-28T11:20:38.978 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-11-28T11:20:39.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:20:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.213+0000 7f0a61dff700 1 Processor -- start 2023-11-28T11:20:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.213+0000 7f0a61dff700 1 -- start start 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 0x7f0a5c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c0d2320 0x7f0a5c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0b5800 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0d1950 con 0x7f0a5c161340 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a61dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0b5520 con 0x7f0a5c0d2320 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a5b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a5b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44864/0 (socket says 172.21.15.53:44864) 2023-11-28T11:20:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a5b7fe700 1 -- 172.21.15.53:0/734443225 learned_addr learned my addr 172.21.15.53:0/734443225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a5bfff700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c0d2320 0x7f0a5c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.214+0000 7f0a5affd700 1 --2- 172.21.15.53:0/734443225 >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 0x7f0a5c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 -- 172.21.15.53:0/734443225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c0d2320 msgr2=0x7f0a5c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c0d2320 0x7f0a5c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 -- 172.21.15.53:0/734443225 >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 msgr2=0x7f0a5c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 --2- 172.21.15.53:0/734443225 >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 0x7f0a5c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 -- 172.21.15.53:0/734443225 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0d1300 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5affd700 1 --2- 172.21.15.53:0/734443225 >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 0x7f0a5c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:39.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5b7fe700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0685c0 tx=0x7f0a4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5870c628c601a42b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5a7fc700 1 -- 172.21.15.53:0/734443225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c011830 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5a7fc700 1 -- 172.21.15.53:0/734443225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c011990 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.215+0000 7f0a5a7fc700 1 -- 172.21.15.53:0/734443225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c011d20 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 -- 172.21.15.53:0/734443225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c15c9b0 msgr2=0x7f0a5c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0685c0 tx=0x7f0a4c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 -- 172.21.15.53:0/734443225 shutdown_connections 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c0d2320 0x7f0a5c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 --2- 172.21.15.53:0/734443225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c15c9b0 0x7f0a5c0b4bf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 --2- 172.21.15.53:0/734443225 >> v2:172.21.15.157:3300/0 conn(0x7f0a5c161340 0x7f0a5c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 -- 172.21.15.53:0/734443225 >> 172.21.15.53:0/734443225 conn(0x7f0a5c072600 msgr2=0x7f0a5c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:39.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 -- 172.21.15.53:0/734443225 shutdown_connections 2023-11-28T11:20:39.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.216+0000 7f0a61dff700 1 -- 172.21.15.53:0/734443225 wait complete. 2023-11-28T11:20:39.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.218+0000 7f0a61dff700 1 Processor -- start 2023-11-28T11:20:39.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.218+0000 7f0a61dff700 1 -- start start 2023-11-28T11:20:39.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a5c0d2320 0x7f0a5c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c15c9b0 0x7f0a5c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c150760 con 0x7f0a5c161340 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1f3680 con 0x7f0a5c0d2320 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a61dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1f2a70 con 0x7f0a5c15c9b0 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a5c0d2320 0x7f0a5c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44870/0 (socket says 172.21.15.53:44870) 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5bfff700 1 -- 172.21.15.53:0/1485332506 learned_addr learned my addr 172.21.15.53:0/1485332506 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5affd700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c15c9b0 0x7f0a5c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.219+0000 7f0a5bfff700 1 -- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c15c9b0 msgr2=0x7f0a5c150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a5bfff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c15c9b0 0x7f0a5c150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a5bfff700 1 -- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a5c0d2320 msgr2=0x7f0a5c1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a5bfff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a5c0d2320 0x7f0a5c1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a5bfff700 1 -- 172.21.15.53:0/1485332506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c009550 con 0x7f0a5c161340 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a5bfff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400efc0 tx=0x7f0a54009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54009b20 con 0x7f0a5c161340 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5400f040 con 0x7f0a5c161340 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c0b5130 con 0x7f0a5c161340 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400cc90 con 0x7f0a5c161340 2023-11-28T11:20:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.220+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0d15e0 con 0x7f0a5c161340 2023-11-28T11:20:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.221+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c0b52c0 con 0x7f0a5c161340 2023-11-28T11:20:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.222+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a54014480 con 0x7f0a5c161340 2023-11-28T11:20:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.223+0000 7f0a43fff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 0x7f0a4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.223+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0a540865a0 con 0x7f0a5c161340 2023-11-28T11:20:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.223+0000 7f0a5b7fe700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 0x7f0a4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.224+0000 7f0a5b7fe700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 0x7f0a4806a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009c10 tx=0x7f0a4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.227+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a54052fd0 con 0x7f0a5c161340 2023-11-28T11:20:39.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.430+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0a5c0745c0 con 0x7f0a5c161340 2023-11-28T11:20:39.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.434+0000 7f0a43fff700 1 -- 172.21.15.53:0/1485332506 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a540565f0 con 0x7f0a5c161340 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.436+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 msgr2=0x7f0a4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.436+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 0x7f0a4806a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009c10 tx=0x7f0a4c008030 comp rx=0 tx=0).stop 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.436+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 msgr2=0x7f0a5c15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.436+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400efc0 tx=0x7f0a54009f60 comp rx=0 tx=0).stop 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 shutdown_connections 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0a480682f0 0x7f0a4806a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a5c161340 0x7f0a5c15ec40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a5c0d2320 0x7f0a5c1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 --2- 172.21.15.53:0/1485332506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a5c15c9b0 0x7f0a5c150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 >> 172.21.15.53:0/1485332506 conn(0x7f0a5c072600 msgr2=0x7f0a5c15a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:39.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 shutdown_connections 2023-11-28T11:20:39.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.437+0000 7f0a61dff700 1 -- 172.21.15.53:0/1485332506 wait complete. 2023-11-28T11:20:39.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:20:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.657+0000 7f56109ff700 1 Processor -- start 2023-11-28T11:20:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.657+0000 7f56109ff700 1 -- start start 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f56109ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f560c0cd7c0 0x7f560c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f56109ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c0d1950 con 0x7f560c161340 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f56109ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c0ce140 con 0x7f560c15c9b0 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f56109ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0b5800 con 0x7f560c0cd7c0 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f5603fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f560c161340 0x7f560c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38178/0 (socket says 172.21.15.53:38178) 2023-11-28T11:20:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.658+0000 7f5603fff700 1 -- 172.21.15.53:0/2310692965 learned_addr learned my addr 172.21.15.53:0/2310692965 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f560ad9d700 1 --2- 172.21.15.53:0/2310692965 >> v2:172.21.15.157:3300/0 conn(0x7f560c0cd7c0 0x7f560c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936153470 0 0) 0x7f560c0d1950 con 0x7f560c161340 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f80035b0 con 0x7f560c161340 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1091626816 0 0) 0x7f55f80035b0 con 0x7f560c161340 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 >> v2:172.21.15.157:3300/0 conn(0x7f560c0cd7c0 msgr2=0x7f560c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 --2- 172.21.15.53:0/2310692965 >> v2:172.21.15.157:3300/0 conn(0x7f560c0cd7c0 0x7f560c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 >> v1:172.21.15.157:6789/0 conn(0x7f560c15c9b0 legacy=0x7f560c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0d24f0 con 0x7f560c161340 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f55f4001670 con 0x7f560c161340 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 >> v1:172.21.15.53:6789/0 conn(0x7f560c161340 legacy=0x7f560c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8005200 0x7f55f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0b5800 con 0x7f55f8003a00 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f800c110 con 0x7f55f8004850 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.659+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f800c2d0 con 0x7f55f8005200 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f5603fff700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560ad9d700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8005200 0x7f55f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 -- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8005200 msgr2=0x7f55f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8005200 0x7f55f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 -- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f8004850 msgr2=0x7f55f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 -- 172.21.15.53:0/2310692965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80035b0 con 0x7f55f8003a00 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560ad9d700 1 -- 172.21.15.53:0/2310692965 reap_dead start 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f560a59c700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0017c0 tx=0x7f55f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f2efc79cedb691 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0052d0 con 0x7f55f8003a00 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc005430 con 0x7f55f8003a00 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.660+0000 7f5609d9b700 1 -- 172.21.15.53:0/2310692965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0038c0 con 0x7f55f8003a00 2023-11-28T11:20:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 -- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 msgr2=0x7f55f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0017c0 tx=0x7f55f8005880 comp rx=0 tx=0).stop 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 -- 172.21.15.53:0/2310692965 shutdown_connections 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 --2- 172.21.15.53:0/2310692965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 -- 172.21.15.53:0/2310692965 >> 172.21.15.53:0/2310692965 conn(0x7f560c072600 msgr2=0x7f560c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 -- 172.21.15.53:0/2310692965 shutdown_connections 2023-11-28T11:20:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.661+0000 7f56109ff700 1 -- 172.21.15.53:0/2310692965 wait complete. 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.663+0000 7f56109ff700 1 Processor -- start 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.663+0000 7f56109ff700 1 -- start start 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8003a00 0x7f560c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 0x7f560c1f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0cbac0 con 0x7f560c153d30 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1f59a0 con 0x7f560c0cadc0 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f56109ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c157e50 con 0x7f55f8003a00 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f560ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f560ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44886/0 (socket says 172.21.15.53:44886) 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f560ad9d700 1 -- 172.21.15.53:0/3820351310 learned_addr learned my addr 172.21.15.53:0/3820351310 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f560a59c700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8003a00 0x7f560c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.664+0000 7f5603fff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 0x7f560c1f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 -- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8003a00 msgr2=0x7f560c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8003a00 0x7f560c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 -- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 msgr2=0x7f560c1f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 0x7f560c1f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 -- 172.21.15.53:0/3820351310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0d1300 con 0x7f560c153d30 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f5603fff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 0x7f560c1f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f560ad9d700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f560c08bfd0 tx=0x7f56040111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400e070 con 0x7f560c153d30 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604003cf0 con 0x7f560c153d30 2023-11-28T11:20:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c0b5520 con 0x7f560c153d30 2023-11-28T11:20:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604011850 con 0x7f560c153d30 2023-11-28T11:20:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.665+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c0d15e0 con 0x7f560c153d30 2023-11-28T11:20:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.666+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0745c0 con 0x7f560c153d30 2023-11-28T11:20:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.667+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f56040119b0 con 0x7f560c153d30 2023-11-28T11:20:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.668+0000 7f5602ffd700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.668+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5604085ec0 con 0x7f560c153d30 2023-11-28T11:20:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.669+0000 7f560a59c700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.669+0000 7f560a59c700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 0x7f55ec06a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55fc015040 tx=0x7f55fc009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.672+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f56040528f0 con 0x7f560c153d30 2023-11-28T11:20:39.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.954+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f560c0d08a0 con 0x7f560c153d30 2023-11-28T11:20:39.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.967+0000 7f5602ffd700 1 -- 172.21.15.53:0/3820351310 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v163) v1 ==== 149+0+0 (secure 0 0 0) 0x7f5604055f10 con 0x7f560c153d30 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 msgr2=0x7f55ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 0x7f55ec06a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55fc015040 tx=0x7f55fc009030 comp rx=0 tx=0).stop 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 msgr2=0x7f560c15e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f560c08bfd0 tx=0x7f56040111d0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 shutdown_connections 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55f8003a00 0x7f560c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f560c153d30 0x7f560c15e7d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 --2- 172.21.15.53:0/3820351310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0cadc0 0x7f560c1f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:39.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 >> 172.21.15.53:0/3820351310 conn(0x7f560c072600 msgr2=0x7f560c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:39.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 shutdown_connections 2023-11-28T11:20:39.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:39.970+0000 7f56109ff700 1 -- 172.21.15.53:0/3820351310 wait complete. 2023-11-28T11:20:39.989 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:20:39.990 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:20:39.990 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:20:39.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:20:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.192+0000 7f0c337ff700 1 Processor -- start 2023-11-28T11:20:40.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.192+0000 7f0c337ff700 1 -- start start 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c2c161340 0x7f0c2c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b5800 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d1950 con 0x7f0c2c161340 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.193+0000 7f0c337ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b5520 con 0x7f0c2c0d2320 2023-11-28T11:20:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44890/0 (socket says 172.21.15.53:44890) 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 -- 172.21.15.53:0/3043775806 learned_addr learned my addr 172.21.15.53:0/3043775806 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c30d9a700 1 --2- 172.21.15.53:0/3043775806 >> v2:172.21.15.157:3300/0 conn(0x7f0c2c161340 0x7f0c2c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c31d9c700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 -- 172.21.15.53:0/3043775806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 msgr2=0x7f0c2c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 -- 172.21.15.53:0/3043775806 >> v2:172.21.15.157:3300/0 conn(0x7f0c2c161340 msgr2=0x7f0c2c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 --2- 172.21.15.53:0/3043775806 >> v2:172.21.15.157:3300/0 conn(0x7f0c2c161340 0x7f0c2c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 -- 172.21.15.53:0/3043775806 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0d1300 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.194+0000 7f0c3159b700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0685c0 tx=0x7f0c28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b45bf91f36a94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c23fff700 1 -- 172.21.15.53:0/3043775806 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800b870 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c23fff700 1 -- 172.21.15.53:0/3043775806 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2800b9d0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c23fff700 1 -- 172.21.15.53:0/3043775806 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800bd60 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 -- 172.21.15.53:0/3043775806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 msgr2=0x7f0c2c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0685c0 tx=0x7f0c28009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 -- 172.21.15.53:0/3043775806 shutdown_connections 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 --2- 172.21.15.53:0/3043775806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c0b4bf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 --2- 172.21.15.53:0/3043775806 >> v2:172.21.15.157:3300/0 conn(0x7f0c2c161340 0x7f0c2c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 -- 172.21.15.53:0/3043775806 >> 172.21.15.53:0/3043775806 conn(0x7f0c2c072600 msgr2=0x7f0c2c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 -- 172.21.15.53:0/3043775806 shutdown_connections 2023-11-28T11:20:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.195+0000 7f0c337ff700 1 -- 172.21.15.53:0/3043775806 wait complete. 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.197+0000 7f0c337ff700 1 Processor -- start 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.197+0000 7f0c337ff700 1 -- start start 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c2c161340 0x7f0c2c15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1f32d0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1f3570 con 0x7f0c2c161340 2023-11-28T11:20:40.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c337ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1f3850 con 0x7f0c2c0d2320 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c30d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c3159b700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c30d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44906/0 (socket says 172.21.15.53:44906) 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c30d9a700 1 -- 172.21.15.53:0/2776352110 learned_addr learned my addr 172.21.15.53:0/2776352110 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.198+0000 7f0c31d9c700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c2c161340 0x7f0c2c15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 -- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 msgr2=0x7f0c2c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 -- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c2c161340 msgr2=0x7f0c2c15f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c2c161340 0x7f0c2c15f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 -- 172.21.15.53:0/2776352110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c28009550 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c3159b700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c30d9a700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00e810 tx=0x7f0c1c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00c070 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c013610 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0139c0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.199+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c0b5130 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.200+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c0d15e0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.200+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c0b52c0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.201+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c1c023070 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.202+0000 7f0c21ffb700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 0x7f0c1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.202+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c1c085790 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.203+0000 7f0c3159b700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 0x7f0c1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.204+0000 7f0c3159b700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 0x7f0c1006a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c28000c00 tx=0x7f0c28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.206+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c1c0521a0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.492+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f0c2c0745c0 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.969+0000 7f0c21ffb700 1 -- 172.21.15.53:0/2776352110 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v164) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c1c00f040 con 0x7f0c2c15c9b0 2023-11-28T11:20:40.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 msgr2=0x7f0c1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 0x7f0c1006a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c28000c00 tx=0x7f0c28008030 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 msgr2=0x7f0c2c15ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00e810 tx=0x7f0c1c00eb20 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 shutdown_connections 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0c100682f0 0x7f0c1006a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c2c0d2320 0x7f0c2c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c2c161340 0x7f0c2c15f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 --2- 172.21.15.53:0/2776352110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c15c9b0 0x7f0c2c15ec20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 >> 172.21.15.53:0/2776352110 conn(0x7f0c2c072600 msgr2=0x7f0c2c15a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 shutdown_connections 2023-11-28T11:20:40.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:40.972+0000 7f0c337ff700 1 -- 172.21.15.53:0/2776352110 wait complete. 2023-11-28T11:20:40.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:20:41.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.203+0000 7f8c213ff700 1 Processor -- start 2023-11-28T11:20:41.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.203+0000 7f8c213ff700 1 -- start start 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c213ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c213ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c1c0cd7c0 0x7f8c1c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c213ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0d1950 con 0x7f8c1c161340 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c213ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b5800 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c213ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0ce140 con 0x7f8c1c0cd7c0 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c1affd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c1affd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43934/0 (socket says 172.21.15.53:43934) 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.204+0000 7f8c1affd700 1 -- 172.21.15.53:0/57313729 learned_addr learned my addr 172.21.15.53:0/57313729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:41.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1b7fe700 1 --2- 172.21.15.53:0/57313729 >> v2:172.21.15.157:3300/0 conn(0x7f8c1c0cd7c0 0x7f8c1c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1affd700 1 -- 172.21.15.53:0/57313729 >> v2:172.21.15.157:3300/0 conn(0x7f8c1c0cd7c0 msgr2=0x7f8c1c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1affd700 1 --2- 172.21.15.53:0/57313729 >> v2:172.21.15.157:3300/0 conn(0x7f8c1c0cd7c0 0x7f8c1c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1affd700 1 -- 172.21.15.53:0/57313729 >> v1:172.21.15.157:6789/0 conn(0x7f8c1c161340 legacy=0x7f8c1c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1affd700 1 -- 172.21.15.53:0/57313729 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0d24f0 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1affd700 1 --2- 172.21.15.53:0/57313729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c06c170 tx=0x7f8c0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf3b017c78d8d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1a7fc700 1 -- 172.21.15.53:0/57313729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00b870 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1a7fc700 1 -- 172.21.15.53:0/57313729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00b9d0 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.205+0000 7f8c1a7fc700 1 -- 172.21.15.53:0/57313729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00bd60 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 -- 172.21.15.53:0/57313729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c15c9b0 msgr2=0x7f8c1c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 --2- 172.21.15.53:0/57313729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c06c170 tx=0x7f8c0c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 -- 172.21.15.53:0/57313729 shutdown_connections 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 --2- 172.21.15.53:0/57313729 >> v2:172.21.15.157:3300/0 conn(0x7f8c1c0cd7c0 0x7f8c1c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 --2- 172.21.15.53:0/57313729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c15c9b0 0x7f8c1c0b4bf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 -- 172.21.15.53:0/57313729 >> 172.21.15.53:0/57313729 conn(0x7f8c1c072600 msgr2=0x7f8c1c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:41.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 -- 172.21.15.53:0/57313729 shutdown_connections 2023-11-28T11:20:41.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.206+0000 7f8c213ff700 1 -- 172.21.15.53:0/57313729 wait complete. 2023-11-28T11:20:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.208+0000 7f8c213ff700 1 Processor -- start 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.208+0000 7f8c213ff700 1 -- start start 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1c0cd7c0 0x7f8c1c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c1c15c9b0 0x7f8c1c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0ca780 con 0x7f8c1c161340 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c156280 con 0x7f8c1c0cd7c0 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c213ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c156560 con 0x7f8c1c15c9b0 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43946/0 (socket says 172.21.15.53:43946) 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c1b7fe700 1 -- 172.21.15.53:0/2338114301 learned_addr learned my addr 172.21.15.53:0/2338114301 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c1affd700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1c0cd7c0 0x7f8c1c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.209+0000 7f8c13fff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c1c15c9b0 0x7f8c1c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 -- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c1c15c9b0 msgr2=0x7f8c1c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c1c15c9b0 0x7f8c1c157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 -- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1c0cd7c0 msgr2=0x7f8c1c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1c0cd7c0 0x7f8c1c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 -- 172.21.15.53:0/2338114301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009550 con 0x7f8c1c161340 2023-11-28T11:20:41.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c1b7fe700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007e60 tx=0x7f8c1400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400ed30 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c14014420 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0d1300 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140147d0 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.210+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c0b5520 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.211+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0d15e0 con 0x7f8c1c161340 2023-11-28T11:20:41.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.212+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c14012050 con 0x7f8c1c161340 2023-11-28T11:20:41.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.214+0000 7f8c137fe700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 0x7f8bfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:41.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.214+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8c14085f80 con 0x7f8c1c161340 2023-11-28T11:20:41.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.214+0000 7f8c1affd700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 0x7f8bfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:41.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.215+0000 7f8c1affd700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 0x7f8bfc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c014040 tx=0x7f8c0c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:41.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.218+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c14052a10 con 0x7f8c1c161340 2023-11-28T11:20:41.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.422+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8c1c159a70 con 0x7f8c1c161340 2023-11-28T11:20:41.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.426+0000 7f8c137fe700 1 -- 172.21.15.53:0/2338114301 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c14056030 con 0x7f8c1c161340 2023-11-28T11:20:41.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 msgr2=0x7f8bfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 0x7f8bfc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c014040 tx=0x7f8c0c008030 comp rx=0 tx=0).stop 2023-11-28T11:20:41.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 msgr2=0x7f8c1c14a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:41.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007e60 tx=0x7f8c1400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 shutdown_connections 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f8bfc0682f0 0x7f8bfc06a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c1c161340 0x7f8c1c14a480 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1c0cd7c0 0x7f8c1c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 --2- 172.21.15.53:0/2338114301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c1c15c9b0 0x7f8c1c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 >> 172.21.15.53:0/2338114301 conn(0x7f8c1c072600 msgr2=0x7f8c1c159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 shutdown_connections 2023-11-28T11:20:41.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:41.428+0000 7f8c213ff700 1 -- 172.21.15.53:0/2338114301 wait complete. 2023-11-28T11:20:41.447 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:20:41.447 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:20:41.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:20:41.634 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:20:41.634 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:41.634 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:41.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:20:41.677 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:20:41.678 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:41.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:20:41.722 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:41.722 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:41.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:20:41.767 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:20:41.767 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:41.810 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:20:41.810 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:20:41.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:42.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.013+0000 7f08c1fff700 1 Processor -- start 2023-11-28T11:20:42.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.013+0000 7f08c1fff700 1 -- start start 2023-11-28T11:20:42.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08c1fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08bc161340 0x7f08bc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08c1fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0d1950 con 0x7f08bc0d2320 2023-11-28T11:20:42.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08c1fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0b5520 con 0x7f08bc15c9b0 2023-11-28T11:20:42.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08c1fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc0b5800 con 0x7f08bc161340 2023-11-28T11:20:42.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08bbfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f08bc0d2320 0x7f08bc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33862/0 (socket says 172.21.15.53:33862) 2023-11-28T11:20:42.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.014+0000 7f08bbfff700 1 -- 172.21.15.53:0/2167350412 learned_addr learned my addr 172.21.15.53:0/2167350412 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:42.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08baffd700 1 --2- 172.21.15.53:0/2167350412 >> v2:172.21.15.157:3300/0 conn(0x7f08bc161340 0x7f08bc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036259802 0 0) 0x7f08bc0d1950 con 0x7f08bc0d2320 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a40035b0 con 0x7f08bc0d2320 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453589306 0 0) 0x7f08bc0b5520 con 0x7f08bc15c9b0 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc0d1950 con 0x7f08bc15c9b0 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3117471587 0 0) 0x7f08a40035b0 con 0x7f08bc0d2320 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 >> v2:172.21.15.157:3300/0 conn(0x7f08bc161340 msgr2=0x7f08bc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 --2- 172.21.15.53:0/2167350412 >> v2:172.21.15.157:3300/0 conn(0x7f08bc161340 0x7f08bc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 >> v1:172.21.15.157:6789/0 conn(0x7f08bc15c9b0 legacy=0x7f08bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0d1300 con 0x7f08bc0d2320 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f08b40019f0 con 0x7f08bc0d2320 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 >> v1:172.21.15.53:6789/0 conn(0x7f08bc0d2320 legacy=0x7f08bc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4003b90 0x7f08a4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 0x7f08a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4005200 0x7f08a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b5800 con 0x7f08a4004850 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a400c110 con 0x7f08a4003b90 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.015+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a400c2d0 con 0x7f08a4005200 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 0x7f08a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08bb7fe700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4003b90 0x7f08a4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08bbfff700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4005200 0x7f08a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 -- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4005200 msgr2=0x7f08a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4005200 0x7f08a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 -- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4003b90 msgr2=0x7f08a4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:42.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4003b90 0x7f08a4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 -- 172.21.15.53:0/2167350412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40035b0 con 0x7f08a4004850 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08bbfff700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4005200 0x7f08a400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08bbfff700 1 -- 172.21.15.53:0/2167350412 reap_dead start 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.016+0000 7f08baffd700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 0x7f08a4004cc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009b30 tx=0x7f08a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ace9424b5ae828 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000e6c0 con 0x7f08a4004850 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b000e820 con 0x7f08a4004850 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08ba7fc700 1 -- 172.21.15.53:0/2167350412 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000eb90 con 0x7f08a4004850 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 -- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 msgr2=0x7f08a4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 0x7f08a4004cc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009b30 tx=0x7f08a4005880 comp rx=0 tx=0).stop 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 -- 172.21.15.53:0/2167350412 shutdown_connections 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2167350412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a4004850 0x7f08a4004cc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 -- 172.21.15.53:0/2167350412 >> 172.21.15.53:0/2167350412 conn(0x7f08bc072600 msgr2=0x7f08bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 -- 172.21.15.53:0/2167350412 shutdown_connections 2023-11-28T11:20:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.017+0000 7f08c1fff700 1 -- 172.21.15.53:0/2167350412 wait complete. 2023-11-28T11:20:42.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.019+0000 7f08c1fff700 1 Processor -- start 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.019+0000 7f08c1fff700 1 -- start start 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004850 0x7f08bc1e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1574b0 0x7f08bc06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b87e0 con 0x7f08bc1574b0 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1e7000 con 0x7f08a4004850 2023-11-28T11:20:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08c1fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1e72e0 con 0x7f08bc0b8300 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08bbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1574b0 0x7f08bc06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08baffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08baffd700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:35972/0 (socket says 172.21.15.53:35972) 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08baffd700 1 -- 172.21.15.53:0/2270189724 learned_addr learned my addr 172.21.15.53:0/2270189724 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.020+0000 7f08bb7fe700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004850 0x7f08bc1e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 -- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004850 msgr2=0x7f08bc1e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004850 0x7f08bc1e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 -- 172.21.15.53:0/2270189724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1574b0 msgr2=0x7f08bc06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1574b0 0x7f08bc06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 -- 172.21.15.53:0/2270189724 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc15cd80 con 0x7f08bc0b8300 2023-11-28T11:20:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08baffd700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c0e0 tx=0x7f08b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000e6c0 con 0x7f08bc0b8300 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0003ad0 con 0x7f08bc0b8300 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc0d15e0 con 0x7f08bc0b8300 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.021+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003e80 con 0x7f08bc0b8300 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.022+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15cf10 con 0x7f08bc0b8300 2023-11-28T11:20:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.022+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc0745c0 con 0x7f08bc0b8300 2023-11-28T11:20:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.023+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08b0025070 con 0x7f08bc0b8300 2023-11-28T11:20:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.024+0000 7f08a3fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 0x7f08a806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.024+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f08b0087120 con 0x7f08bc0b8300 2023-11-28T11:20:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.024+0000 7f08bb7fe700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 0x7f08a806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.025+0000 7f08bb7fe700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 0x7f08a806a7d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.029+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08b0053bb0 con 0x7f08bc0b8300 2023-11-28T11:20:42.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.274+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08bc0c6a20 con 0x7f08bc0b8300 2023-11-28T11:20:42.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.276+0000 7f08a3fff700 1 -- 172.21.15.53:0/2270189724 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f08b0017de0 con 0x7f08bc0b8300 2023-11-28T11:20:42.276 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:42.276 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2142515670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2142515670},{"type":"v1","addr":"172.21.15.157:6836","nonce":2142515670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2658545505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2658545505},{"type":"v1","addr":"172.21.15.157:6837","nonce":2658545505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3202905315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202905315},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202905315}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2207775120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2207775120},{"type":"v1","addr":"172.21.15.53:6835","nonce":2207775120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 msgr2=0x7f08a806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 0x7f08a806a7d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 msgr2=0x7f08bc156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c0e0 tx=0x7f08b00047c0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 shutdown_connections 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f08a8068320 0x7f08a806a7d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1574b0 0x7f08bc06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004850 0x7f08bc1e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 --2- 172.21.15.53:0/2270189724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08bc0b8300 0x7f08bc156c20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 >> 172.21.15.53:0/2270189724 conn(0x7f08bc072600 msgr2=0x7f08bc15fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:42.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 shutdown_connections 2023-11-28T11:20:42.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.278+0000 7f08c1fff700 1 -- 172.21.15.53:0/2270189724 wait complete. 2023-11-28T11:20:42.282 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2023-11-28T11:20:42.297 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:20:42.297 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:20:42.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:20:42.298 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:20:42.298 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:20:42.298 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:20:42.298 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:20:42.298 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:20:42.298 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:20:42.299 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:20:42.299 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:20:42.299 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:20:42.299 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:20:42.299 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:20:42.299 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:20:42.299 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:20:42.315 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:20:42.315 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:20:42.315 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:20:42.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:20:42.317 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:20:42.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:20:42.317 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:20:42.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:20:42.359 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:20:42.360 DEBUG:teuthology.parallel:result is None 2023-11-28T11:20:42.361 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:20:42.361 DEBUG:teuthology.parallel:result is None 2023-11-28T11:20:42.362 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:20:42.362 DEBUG:teuthology.parallel:result is None 2023-11-28T11:20:42.364 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:20:42.364 DEBUG:teuthology.parallel:result is None 2023-11-28T11:20:42.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:20:42.505 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-11-28T11:20:42.505 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-11-28T11:20:42.514 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-11-28T11:20:42.514 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:20:42.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.578+0000 7ff3279ff700 1 Processor -- start 2023-11-28T11:20:42.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.578+0000 7ff3279ff700 1 -- start start 2023-11-28T11:20:42.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff3279ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 0x7ff3200b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff3279ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 0x7ff3200cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff3279ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200ce160 con 0x7ff320161340 2023-11-28T11:20:42.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff3279ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200b5800 con 0x7ff3200cd7c0 2023-11-28T11:20:42.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff3279ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200d1950 con 0x7ff32015c9b0 2023-11-28T11:20:42.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff324f9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff320161340 0x7ff3200d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33866/0 (socket says 172.21.15.53:33866) 2023-11-28T11:20:42.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.579+0000 7ff324f9a700 1 -- 172.21.15.53:0/1629796700 learned_addr learned my addr 172.21.15.53:0/1629796700 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:42.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 --2- 172.21.15.53:0/1629796700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 0x7ff3200b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff325f9c700 1 --2- 172.21.15.53:0/1629796700 >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 0x7ff3200cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff317fff700 1 -- 172.21.15.53:0/1629796700 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406371396 0 0) 0x7ff3200ce160 con 0x7ff320161340 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff317fff700 1 -- 172.21.15.53:0/1629796700 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff30c0035b0 con 0x7ff320161340 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 -- 172.21.15.53:0/1629796700 >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 msgr2=0x7ff3200cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 --2- 172.21.15.53:0/1629796700 >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 0x7ff3200cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 -- 172.21.15.53:0/1629796700 >> v1:172.21.15.53:6789/0 conn(0x7ff320161340 legacy=0x7ff3200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 -- 172.21.15.53:0/1629796700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200d24f0 con 0x7ff32015c9b0 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff325f9c700 1 --2- 172.21.15.53:0/1629796700 >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 0x7ff3200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.580+0000 7ff32579b700 1 --2- 172.21.15.53:0/1629796700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 0x7ff3200b4bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009840 tx=0x7ff31c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a26cc2969c41a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff317fff700 1 -- 172.21.15.53:0/1629796700 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011830 con 0x7ff32015c9b0 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff317fff700 1 -- 172.21.15.53:0/1629796700 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c011990 con 0x7ff32015c9b0 2023-11-28T11:20:42.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff317fff700 1 -- 172.21.15.53:0/1629796700 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011d00 con 0x7ff32015c9b0 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 -- 172.21.15.53:0/1629796700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 msgr2=0x7ff3200b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 --2- 172.21.15.53:0/1629796700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 0x7ff3200b4bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009840 tx=0x7ff31c009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 -- 172.21.15.53:0/1629796700 shutdown_connections 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 --2- 172.21.15.53:0/1629796700 >> v2:172.21.15.157:3300/0 conn(0x7ff3200cd7c0 0x7ff3200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 --2- 172.21.15.53:0/1629796700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff32015c9b0 0x7ff3200b4bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.581+0000 7ff3279ff700 1 -- 172.21.15.53:0/1629796700 >> 172.21.15.53:0/1629796700 conn(0x7ff320072600 msgr2=0x7ff3200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.582+0000 7ff3279ff700 1 -- 172.21.15.53:0/1629796700 shutdown_connections 2023-11-28T11:20:42.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.582+0000 7ff3279ff700 1 -- 172.21.15.53:0/1629796700 wait complete. 2023-11-28T11:20:42.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.583+0000 7ff3279ff700 1 Processor -- start 2023-11-28T11:20:42.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.584+0000 7ff3279ff700 1 -- start start 2023-11-28T11:20:42.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.584+0000 7ff3279ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 0x7ff3200c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.584+0000 7ff3279ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.584+0000 7ff3279ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff320161340 0x7ff32014b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff3279ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200c7a80 con 0x7ff32015c9b0 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff3279ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200c7d60 con 0x7ff3200cd7c0 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff3279ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200c8040 con 0x7ff320161340 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43988/0 (socket says 172.21.15.53:43988) 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 -- 172.21.15.53:0/128524092 learned_addr learned my addr 172.21.15.53:0/128524092 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff32579b700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 0x7ff3200c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff325f9c700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff320161340 0x7ff32014b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 -- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff320161340 msgr2=0x7ff32014b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff320161340 0x7ff32014b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 -- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 msgr2=0x7ff3200c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 0x7ff3200c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff324f9a700 1 -- 172.21.15.53:0/128524092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009a70 con 0x7ff32015c9b0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.585+0000 7ff32579b700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 0x7ff3200c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff324f9a700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c420 tx=0x7ff31000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310005d70 con 0x7ff32015c9b0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31000ce20 con 0x7ff32015c9b0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c0094a0 con 0x7ff32015c9b0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100072d0 con 0x7ff32015c9b0 2023-11-28T11:20:42.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.586+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3200d1300 con 0x7ff32015c9b0 2023-11-28T11:20:42.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.587+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3200b5520 con 0x7ff32015c9b0 2023-11-28T11:20:42.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.588+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff310019030 con 0x7ff32015c9b0 2023-11-28T11:20:42.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.589+0000 7ff315ffb700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 0x7ff30406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:42.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.589+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff310084b40 con 0x7ff32015c9b0 2023-11-28T11:20:42.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.590+0000 7ff32579b700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 0x7ff30406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:42.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.590+0000 7ff32579b700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 0x7ff30406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:42.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.593+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff31004b5e0 con 0x7ff32015c9b0 2023-11-28T11:20:42.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:42.810+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3200d15e0 con 0x7ff32015c9b0 2023-11-28T11:20:43.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.536+0000 7ff315ffb700 1 -- 172.21.15.53:0/128524092 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff310017020 con 0x7ff32015c9b0 2023-11-28T11:20:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 msgr2=0x7ff30406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 0x7ff30406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c01c040 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 msgr2=0x7ff3200c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c420 tx=0x7ff31000c730 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 shutdown_connections 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff3040682f0 0x7ff30406a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff320161340 0x7ff32014b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3200cd7c0 0x7ff3200c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 --2- 172.21.15.53:0/128524092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32015c9b0 0x7ff3200c7250 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 >> 172.21.15.53:0/128524092 conn(0x7ff320072600 msgr2=0x7ff3200d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 shutdown_connections 2023-11-28T11:20:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.539+0000 7ff3279ff700 1 -- 172.21.15.53:0/128524092 wait complete. 2023-11-28T11:20:43.543 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-11-28T11:20:43.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:20:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.764+0000 7fd336dff700 1 Processor -- start 2023-11-28T11:20:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.764+0000 7fd336dff700 1 -- start start 2023-11-28T11:20:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd336dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330161340 0x7fd3300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd336dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3300d2320 0x7fd3300cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd336dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3300d1950 con 0x7fd33015c9b0 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd336dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300b5800 con 0x7fd3300d2320 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd336dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300b5300 con 0x7fd330161340 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd33539c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3300d2320 0x7fd3300cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd33539c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3300d2320 0x7fd3300cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44004/0 (socket says 172.21.15.53:44004) 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd33539c700 1 -- 172.21.15.53:0/1625845216 learned_addr learned my addr 172.21.15.53:0/1625845216 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.765+0000 7fd32ffff700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330161340 0x7fd3300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd33539c700 1 -- 172.21.15.53:0/1625845216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330161340 msgr2=0x7fd3300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd33539c700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330161340 0x7fd3300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd33539c700 1 -- 172.21.15.53:0/1625845216 >> v1:172.21.15.157:6789/0 conn(0x7fd33015c9b0 legacy=0x7fd3300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd33539c700 1 -- 172.21.15.53:0/1625845216 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300b5520 con 0x7fd3300d2320 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd33539c700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3300d2320 0x7fd3300cd7c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e80 tx=0x7fd32800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b093ac5e9be6f121 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd32f7fe700 1 -- 172.21.15.53:0/1625845216 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800e070 con 0x7fd3300d2320 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd32f7fe700 1 -- 172.21.15.53:0/1625845216 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800b980 con 0x7fd3300d2320 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.766+0000 7fd32f7fe700 1 -- 172.21.15.53:0/1625845216 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800bd30 con 0x7fd3300d2320 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 -- 172.21.15.53:0/1625845216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3300d2320 msgr2=0x7fd3300cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3300d2320 0x7fd3300cd7c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e80 tx=0x7fd32800b330 comp rx=0 tx=0).stop 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 -- 172.21.15.53:0/1625845216 shutdown_connections 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3300d2320 0x7fd3300cd7c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 --2- 172.21.15.53:0/1625845216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330161340 0x7fd3300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 -- 172.21.15.53:0/1625845216 >> 172.21.15.53:0/1625845216 conn(0x7fd330072600 msgr2=0x7fd3300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 -- 172.21.15.53:0/1625845216 shutdown_connections 2023-11-28T11:20:43.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.767+0000 7fd336dff700 1 -- 172.21.15.53:0/1625845216 wait complete. 2023-11-28T11:20:43.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.769+0000 7fd336dff700 1 Processor -- start 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.769+0000 7fd336dff700 1 -- start start 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 0x7fd3301e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd33015c9b0 0x7fd3301e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c8110 con 0x7fd330161340 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c7490 con 0x7fd3300d2320 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd336dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c7770 con 0x7fd33015c9b0 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd33539c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd33539c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44008/0 (socket says 172.21.15.53:44008) 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd33539c700 1 -- 172.21.15.53:0/1067099188 learned_addr learned my addr 172.21.15.53:0/1067099188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd334b9b700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 0x7fd3301e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.770+0000 7fd32ffff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd33015c9b0 0x7fd3301e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 -- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd33015c9b0 msgr2=0x7fd3301e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd33015c9b0 0x7fd3301e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 -- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 msgr2=0x7fd3301e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 0x7fd3301e8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 -- 172.21.15.53:0/1067099188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009b30 con 0x7fd330161340 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd334b9b700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 0x7fd3301e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd33539c700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd3300675d0 tx=0x7fd328004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328004960 con 0x7fd330161340 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280099b0 con 0x7fd330161340 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3300d15e0 con 0x7fd330161340 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32801b670 con 0x7fd330161340 2023-11-28T11:20:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.771+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3300d1300 con 0x7fd330161340 2023-11-28T11:20:43.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.772+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33015cd80 con 0x7fd330161340 2023-11-28T11:20:43.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.773+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd328021030 con 0x7fd330161340 2023-11-28T11:20:43.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.774+0000 7fd32d7fa700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 0x7fd31406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:43.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.774+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd328086ec0 con 0x7fd330161340 2023-11-28T11:20:43.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.775+0000 7fd334b9b700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 0x7fd31406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:43.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.776+0000 7fd334b9b700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 0x7fd31406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd324000f60 tx=0x7fd324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:43.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:43.779+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd328053a70 con 0x7fd330161340 2023-11-28T11:20:44.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.060+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd3300c8250 con 0x7fd330161340 2023-11-28T11:20:44.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.061+0000 7fd32d7fa700 1 -- 172.21.15.53:0/1067099188 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fd3300c8250 con 0x7fd330161340 2023-11-28T11:20:44.063 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:44.063 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:44.063 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVlNkJINxAAwB6S1pnMrZk3sG6i+8PR9Q==", 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.064 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-11-28T11:20:44.065 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlHGGyFBAAkLNG+TdDzS8Rd+Q2LV6BYw==", 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlH5ZYCBAAdBiB6rthc32/VYiX/UUEpg==", 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:20:44.067 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-11-28T11:20:44.068 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlUdNXIRAAete67odjh+SjYr5KUAmDvw==", 2023-11-28T11:20:44.069 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.069 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.069 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:20:44.069 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:20:44.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.071 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-11-28T11:20:44.071 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAjzGVlnYF9HxAAx5MdeV6qs7Ce0YL/BzjUuQ==", 2023-11-28T11:20:44.071 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.071 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.072 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.073 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-11-28T11:20:44.073 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAmzGVlheXPJhAAqVOmYx93P2jBYNhzlsg4EQ==", 2023-11-28T11:20:44.074 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.074 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.075 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.075 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.077 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-11-28T11:20:44.077 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQApzGVlRXhLLhAAcCx5wY3VywAx4UWnz15sOg==", 2023-11-28T11:20:44.077 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.078 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.078 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.078 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.080 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-11-28T11:20:44.080 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAszGVlxc3CNBAAo8dwkn6w3PZYcNFkTdxZEA==", 2023-11-28T11:20:44.080 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.081 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.081 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.081 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAxzGVlKUZYJRAALIbrf8b2mXrVTt8ebMFSnA==", 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-11-28T11:20:44.083 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA0zGVl2RxXKRAAyHpuvnKEAS2o9BNE1LdRGw==", 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA3zGVlFw1WLRAAf+SrDy5UyYmdT3NfXFYA2A==", 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQA6zGVlCqCMMBAAq6pur3Ycnp4CiTD40y4upw==", 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.087 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlvIIFLBAAxrqCs6enHKg93ih+QHQcdw==", 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:20:44.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.089 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-11-28T11:20:44.089 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVlkkNgABAAD7UONIytbl4yWdjvGODhXw==", 2023-11-28T11:20:44.089 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAgzGVlrXsWNBAA+cEN015xjMYKa6JKavZobQ==", 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:20:44.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-11-28T11:20:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAhzGVl2rlMCBAA0jfD14UyD2AjgyjtkShrTw==", 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.094 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-11-28T11:20:44.094 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAezGVlf022ORAA2Fv9xeE7q2Pkr4B6c7ncyA==", 2023-11-28T11:20:44.094 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVllCcfLxAAjmT46vM+3CEOyE6aCbidrQ==", 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQAfzGVl91F6JhAAGc79E5/TfWduB198lkdQqw==", 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-11-28T11:20:44.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-11-28T11:20:44.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-11-28T11:20:44.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:44.098 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-28T11:20:44.098 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:44.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 msgr2=0x7fd31406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 0x7fd31406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd324000f60 tx=0x7fd324009040 comp rx=0 tx=0).stop 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 msgr2=0x7fd3300c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd3300675d0 tx=0x7fd328004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 shutdown_connections 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd3140682f0 0x7fd31406a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd330161340 0x7fd3300c5e80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300d2320 0x7fd3301e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 --2- 172.21.15.53:0/1067099188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd33015c9b0 0x7fd3301e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 >> 172.21.15.53:0/1067099188 conn(0x7fd330072600 msgr2=0x7fd33015bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 shutdown_connections 2023-11-28T11:20:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.063+0000 7fd336dff700 1 -- 172.21.15.53:0/1067099188 wait complete. 2023-11-28T11:20:44.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:44.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.304+0000 7f88c1dff700 1 Processor -- start 2023-11-28T11:20:44.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.304+0000 7f88c1dff700 1 -- start start 2023-11-28T11:20:44.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88c1dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88bc15c9b0 0x7f88bc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88c1dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88bc0cd7c0 0x7f88bc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88c1dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc0d1950 con 0x7f88bc161340 2023-11-28T11:20:44.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88c1dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0b5800 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88c1dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0ce140 con 0x7f88bc15c9b0 2023-11-28T11:20:44.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88baffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f88bc161340 0x7f88bc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33880/0 (socket says 172.21.15.53:33880) 2023-11-28T11:20:44.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88bbfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88bc0cd7c0 0x7f88bc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88baffd700 1 -- 172.21.15.53:0/3038184770 learned_addr learned my addr 172.21.15.53:0/3038184770 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:44.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.305+0000 7f88bb7fe700 1 --2- 172.21.15.53:0/3038184770 >> v2:172.21.15.157:3300/0 conn(0x7f88bc15c9b0 0x7f88bc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88bbfff700 1 -- 172.21.15.53:0/3038184770 >> v2:172.21.15.157:3300/0 conn(0x7f88bc15c9b0 msgr2=0x7f88bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:44.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88bbfff700 1 --2- 172.21.15.53:0/3038184770 >> v2:172.21.15.157:3300/0 conn(0x7f88bc15c9b0 0x7f88bc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88bbfff700 1 -- 172.21.15.53:0/3038184770 >> v1:172.21.15.53:6789/0 conn(0x7f88bc161340 legacy=0x7f88bc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88bbfff700 1 -- 172.21.15.53:0/3038184770 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0d24f0 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88bbfff700 1 --2- 172.21.15.53:0/3038184770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc0cdc00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f88bc067460 tx=0x7f88b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256d19cd25be05f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88ba7fc700 1 -- 172.21.15.53:0/3038184770 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400e070 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88ba7fc700 1 -- 172.21.15.53:0/3038184770 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400bb20 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.306+0000 7f88ba7fc700 1 -- 172.21.15.53:0/3038184770 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4012430 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 -- 172.21.15.53:0/3038184770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 msgr2=0x7f88bc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3038184770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc0cdc00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f88bc067460 tx=0x7f88b400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:44.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 -- 172.21.15.53:0/3038184770 shutdown_connections 2023-11-28T11:20:44.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3038184770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc0cdc00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3038184770 >> v2:172.21.15.157:3300/0 conn(0x7f88bc15c9b0 0x7f88bc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 -- 172.21.15.53:0/3038184770 >> 172.21.15.53:0/3038184770 conn(0x7f88bc072600 msgr2=0x7f88bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:44.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 -- 172.21.15.53:0/3038184770 shutdown_connections 2023-11-28T11:20:44.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.307+0000 7f88c1dff700 1 -- 172.21.15.53:0/3038184770 wait complete. 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.309+0000 7f88c1dff700 1 Processor -- start 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.309+0000 7f88c1dff700 1 -- start start 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88bc15c9b0 0x7f88bc1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 0x7f88bc15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc156b80 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc1f2a10 con 0x7f88bc15c9b0 2023-11-28T11:20:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88c1dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c9c60 con 0x7f88bc161340 2023-11-28T11:20:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44028/0 (socket says 172.21.15.53:44028) 2023-11-28T11:20:44.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 -- 172.21.15.53:0/3152161752 learned_addr learned my addr 172.21.15.53:0/3152161752 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:44.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88baffd700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88bc15c9b0 0x7f88bc1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bbfff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 0x7f88bc15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 -- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 msgr2=0x7f88bc15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 0x7f88bc15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 -- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88bc15c9b0 msgr2=0x7f88bc1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88bc15c9b0 0x7f88bc1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bb7fe700 1 -- 172.21.15.53:0/3152161752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4009a70 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.310+0000 7f88bbfff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 0x7f88bc15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88bb7fe700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00e870 tx=0x7f88ac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac009590 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac013680 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0139f0 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc0d1300 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.311+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0b5520 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.312+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc0d15e0 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.313+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88ac019030 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.314+0000 7f88a3fff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 0x7f88a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.314+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f88ac0857b0 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.315+0000 7f88baffd700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 0x7f88a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.315+0000 7f88baffd700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 0x7f88a806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ca1d0 tx=0x7f88b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.318+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88ac052270 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.523+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88bc156e60 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.524+0000 7f88a3fff700 1 -- 172.21.15.53:0/3152161752 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+14267 (secure 0 0 0) 0x7f88ac055890 con 0x7f88bc0cd7c0 2023-11-28T11:20:44.526 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:44.526 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":57,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:20:43.534473+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 msgr2=0x7f88a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 0x7f88a806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ca1d0 tx=0x7f88b0009040 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 msgr2=0x7f88bc14a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00e870 tx=0x7f88ac00eb80 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 shutdown_connections 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f88a80682f0 0x7f88a806a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88bc161340 0x7f88bc15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88bc0cd7c0 0x7f88bc14a510 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.526+0000 7f88c1dff700 1 --2- 172.21.15.53:0/3152161752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88bc15c9b0 0x7f88bc1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.527+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 >> 172.21.15.53:0/3152161752 conn(0x7f88bc072600 msgr2=0x7f88bc15f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.527+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 shutdown_connections 2023-11-28T11:20:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.527+0000 7f88c1dff700 1 -- 172.21.15.53:0/3152161752 wait complete. 2023-11-28T11:20:44.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:20:44.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:20:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.750+0000 7f70417ff700 1 Processor -- start 2023-11-28T11:20:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.750+0000 7f70417ff700 1 -- start start 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.751+0000 7f70417ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.751+0000 7f70417ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f703c0d2320 0x7f703c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.751+0000 7f70417ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c0d1950 con 0x7f703c15c9b0 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.751+0000 7f70417ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703c0b5800 con 0x7f703c0d2320 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.751+0000 7f70417ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0b5300 con 0x7f703c161340 2023-11-28T11:20:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f703c0d2320 0x7f703c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f703c0d2320 0x7f703c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44044/0 (socket says 172.21.15.53:44044) 2023-11-28T11:20:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 -- 172.21.15.53:0/3872722620 learned_addr learned my addr 172.21.15.53:0/3872722620 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703a7fc700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 -- 172.21.15.53:0/3872722620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 msgr2=0x7f703c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 -- 172.21.15.53:0/3872722620 >> v1:172.21.15.157:6789/0 conn(0x7f703c15c9b0 legacy=0x7f703c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 -- 172.21.15.53:0/3872722620 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0b5520 con 0x7f703c0d2320 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.752+0000 7f703b7fe700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c0d2320 0x7f703c0cd7c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7034009e80 tx=0x7f703400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76c7c138902d72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f7039ffb700 1 -- 172.21.15.53:0/3872722620 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400e070 con 0x7f703c0d2320 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f7039ffb700 1 -- 172.21.15.53:0/3872722620 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400b8d0 con 0x7f703c0d2320 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f7039ffb700 1 -- 172.21.15.53:0/3872722620 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400bc80 con 0x7f703c0d2320 2023-11-28T11:20:44.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 -- 172.21.15.53:0/3872722620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c0d2320 msgr2=0x7f703c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c0d2320 0x7f703c0cd7c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7034009e80 tx=0x7f703400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 -- 172.21.15.53:0/3872722620 shutdown_connections 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c0d2320 0x7f703c0cd7c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 --2- 172.21.15.53:0/3872722620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 -- 172.21.15.53:0/3872722620 >> 172.21.15.53:0/3872722620 conn(0x7f703c072600 msgr2=0x7f703c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 -- 172.21.15.53:0/3872722620 shutdown_connections 2023-11-28T11:20:44.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.753+0000 7f70417ff700 1 -- 172.21.15.53:0/3872722620 wait complete. 2023-11-28T11:20:44.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.756+0000 7f70417ff700 1 Processor -- start 2023-11-28T11:20:44.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.756+0000 7f70417ff700 1 -- start start 2023-11-28T11:20:44.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0d2320 0x7f703c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c1f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c6ca0 con 0x7f703c15c9b0 2023-11-28T11:20:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c1f3110 con 0x7f703c0d2320 2023-11-28T11:20:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.757+0000 7f70417ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c1f33f0 con 0x7f703c161340 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44054/0 (socket says 172.21.15.53:44054) 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 -- 172.21.15.53:0/74457442 learned_addr learned my addr 172.21.15.53:0/74457442 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703affd700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0d2320 0x7f703c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703b7fe700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c1f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 -- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 msgr2=0x7f703c1f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c1f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 -- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0d2320 msgr2=0x7f703c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0d2320 0x7f703c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 -- 172.21.15.53:0/74457442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034009b30 con 0x7f703c15c9b0 2023-11-28T11:20:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f703a7fc700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f703000e9b0 tx=0x7f703000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.758+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030009710 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.759+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703000ee50 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.759+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c0d15e0 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.759+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703000ca50 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.759+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c0d1300 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.760+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c15cd80 con 0x7f703c15c9b0 2023-11-28T11:20:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.761+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7030019030 con 0x7f703c15c9b0 2023-11-28T11:20:44.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.762+0000 7f70237fe700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 0x7f702806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:44.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.762+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7030085930 con 0x7f703c15c9b0 2023-11-28T11:20:44.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.763+0000 7f703affd700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 0x7f702806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:44.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.763+0000 7f703affd700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 0x7f702806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f702c007fd0 tx=0x7f702c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:44.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.766+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7030052390 con 0x7f703c15c9b0 2023-11-28T11:20:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.970+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f703c0745c0 con 0x7f703c15c9b0 2023-11-28T11:20:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.971+0000 7f70237fe700 1 -- 172.21.15.53:0/74457442 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7030021020 con 0x7f703c15c9b0 2023-11-28T11:20:44.973 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 msgr2=0x7f702806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 0x7f702806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f702c007fd0 tx=0x7f702c009040 comp rx=0 tx=0).stop 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 msgr2=0x7f703c0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f703000e9b0 tx=0x7f703000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 shutdown_connections 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f70280682f0 0x7f702806a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f703c161340 0x7f703c1f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f703c0d2320 0x7f703c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 --2- 172.21.15.53:0/74457442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f703c15c9b0 0x7f703c0c76f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:44.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 >> 172.21.15.53:0/74457442 conn(0x7f703c072600 msgr2=0x7f703c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:44.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 shutdown_connections 2023-11-28T11:20:44.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:44.973+0000 7f70417ff700 1 -- 172.21.15.53:0/74457442 wait complete. 2023-11-28T11:20:44.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:20:45.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:45.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:45.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.199+0000 7faac05ff700 1 Processor -- start 2023-11-28T11:20:45.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.199+0000 7faac05ff700 1 -- start start 2023-11-28T11:20:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.200+0000 7faac05ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faab80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.200+0000 7faac05ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80d1950 con 0x7faab80d2320 2023-11-28T11:20:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.200+0000 7faac05ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80b5520 con 0x7faab815c9b0 2023-11-28T11:20:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.200+0000 7faac05ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab80b5800 con 0x7faab8161340 2023-11-28T11:20:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabeb9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faab80d2320 0x7faab80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33882/0 (socket says 172.21.15.53:33882) 2023-11-28T11:20:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabeb9c700 1 -- 172.21.15.53:0/651956177 learned_addr learned my addr 172.21.15.53:0/651956177 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabdb9a700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faab80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577271612 0 0) 0x7faab80d1950 con 0x7faab80d2320 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa40035b0 con 0x7faab80d2320 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214471844 0 0) 0x7faaa40035b0 con 0x7faab80d2320 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 msgr2=0x7faab80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faab80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 >> v1:172.21.15.157:6789/0 conn(0x7faab815c9b0 legacy=0x7faab80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80d1300 con 0x7faab80d2320 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7faab00019f0 con 0x7faab80d2320 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.201+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 >> v1:172.21.15.53:6789/0 conn(0x7faab80d2320 legacy=0x7faab80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa4003a30 0x7faaa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faaa4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4004d90 0x7faaa4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80b5800 con 0x7faaa4003a30 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa400baf0 con 0x7faaa4004d90 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa400bcb0 con 0x7faab8161340 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa4003a30 0x7faaa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabdb9a700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faaa4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabeb9c700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4004d90 0x7faaa4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 -- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 msgr2=0x7faaa4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab8161340 0x7faaa4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 -- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4004d90 msgr2=0x7faaa4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4004d90 0x7faaa4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.202+0000 7faabe39b700 1 -- 172.21.15.53:0/651956177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa40035b0 con 0x7faaa4003a30 2023-11-28T11:20:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faabe39b700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa4003a30 0x7faaa4003e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faaa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c432718886d4269 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab400c660 con 0x7faaa4003a30 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab400c7e0 con 0x7faaa4003a30 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faabd399700 1 -- 172.21.15.53:0/651956177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab400cb90 con 0x7faaa4003a30 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faac05ff700 1 -- 172.21.15.53:0/651956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa4003a30 msgr2=0x7faaa4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faac05ff700 1 --2- 172.21.15.53:0/651956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa4003a30 0x7faaa4003e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faaa4005880 comp rx=0 tx=0).stop 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faabeb9c700 1 -- 172.21.15.53:0/651956177 reap_dead start 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faac05ff700 1 -- 172.21.15.53:0/651956177 shutdown_connections 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faac05ff700 1 -- 172.21.15.53:0/651956177 >> 172.21.15.53:0/651956177 conn(0x7faab8072600 msgr2=0x7faab80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.203+0000 7faac05ff700 1 -- 172.21.15.53:0/651956177 shutdown_connections 2023-11-28T11:20:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.204+0000 7faac05ff700 1 -- 172.21.15.53:0/651956177 wait complete. 2023-11-28T11:20:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.205+0000 7faac05ff700 1 Processor -- start 2023-11-28T11:20:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.205+0000 7faac05ff700 1 -- start start 2023-11-28T11:20:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4003a30 0x7faab81f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab80cd4a0 0x7faab80cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81f3680 con 0x7faab8157a20 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80cac10 con 0x7faaa4003a30 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faac05ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab8154270 con 0x7faab80cd4a0 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.206+0000 7faabdb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44076/0 (socket says 172.21.15.53:44076) 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 -- 172.21.15.53:0/1862874577 learned_addr learned my addr 172.21.15.53:0/1862874577 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 -- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab80cd4a0 msgr2=0x7faab80cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab80cd4a0 0x7faab80cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 -- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4003a30 msgr2=0x7faab81f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4003a30 0x7faab81f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 -- 172.21.15.53:0/1862874577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab815cd80 con 0x7faab8157a20 2023-11-28T11:20:45.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faabdb9a700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faaa8011a20 tx=0x7faaa8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800f940 con 0x7faab8157a20 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa800faa0 con 0x7faab8157a20 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab80d15e0 con 0x7faab8157a20 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.207+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800fe50 con 0x7faab8157a20 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.208+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab815cf10 con 0x7faab8157a20 2023-11-28T11:20:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.208+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab80745c0 con 0x7faab8157a20 2023-11-28T11:20:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.209+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faaa8014440 con 0x7faab8157a20 2023-11-28T11:20:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.210+0000 7faaaeffd700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 0x7faa9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.210+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faaa80861c0 con 0x7faab8157a20 2023-11-28T11:20:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.211+0000 7faabe39b700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 0x7faa9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.211+0000 7faabe39b700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 0x7faa9c06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.214+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faaa8052ce0 con 0x7faab8157a20 2023-11-28T11:20:45.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.414+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faab80cad80 con 0x7faab8157a20 2023-11-28T11:20:45.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.415+0000 7faaaeffd700 1 -- 172.21.15.53:0/1862874577 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7faaa8056300 con 0x7faab8157a20 2023-11-28T11:20:45.420 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 msgr2=0x7faa9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 0x7faa9c06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4009030 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 msgr2=0x7faab8153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faaa8011a20 tx=0x7faaa8011de0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 shutdown_connections 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faa9c0682f0 0x7faa9c06a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faab80cd4a0 0x7faab80cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaa4003a30 0x7faab81f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 --2- 172.21.15.53:0/1862874577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab8157a20 0x7faab8153d30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 >> 172.21.15.53:0/1862874577 conn(0x7faab8072600 msgr2=0x7faab80b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 shutdown_connections 2023-11-28T11:20:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.417+0000 7faac05ff700 1 -- 172.21.15.53:0/1862874577 wait complete. 2023-11-28T11:20:45.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:20:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.637+0000 7f4e3fdff700 1 Processor -- start 2023-11-28T11:20:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.638+0000 7f4e3fdff700 1 -- start start 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3fdff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e380d1950 con 0x7f4e380cd7c0 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3fdff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e380d24f0 con 0x7f4e38161340 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380b5800 con 0x7f4e3815c9b0 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3e39c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4e380cd7c0 0x7f4e380b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33884/0 (socket says 172.21.15.53:33884) 2023-11-28T11:20:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3e39c700 1 -- 172.21.15.53:0/634246481 learned_addr learned my addr 172.21.15.53:0/634246481 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:45.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778864755 0 0) 0x7f4e380d1950 con 0x7f4e380cd7c0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.639+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e240035b0 con 0x7f4e380cd7c0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562401783 0 0) 0x7f4e380d24f0 con 0x7f4e38161340 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e380d1950 con 0x7f4e38161340 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304776557 0 0) 0x7f4e240035b0 con 0x7f4e380cd7c0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 msgr2=0x7f4e380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e380b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 >> v1:172.21.15.157:6789/0 conn(0x7f4e38161340 legacy=0x7f4e380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380b5520 con 0x7f4e380cd7c0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f4e300019f0 con 0x7f4e380cd7c0 2023-11-28T11:20:45.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e380b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 >> v1:172.21.15.53:6789/0 conn(0x7f4e380cd7c0 legacy=0x7f4e380b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24003bc0 0x7f4e24003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e24004850 0x7f4e24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e24005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380b5800 con 0x7f4e24004850 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2400bb60 con 0x7f4e24003bc0 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2400bd20 con 0x7f4e3815c9b0 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3d39a700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e24004850 0x7f4e24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24003bc0 0x7f4e24003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.640+0000 7f4e3e39c700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e24005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 -- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 msgr2=0x7f4e24005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e3815c9b0 0x7f4e24005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 -- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24003bc0 msgr2=0x7f4e24003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24003bc0 0x7f4e24003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 -- 172.21.15.53:0/634246481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340094a0 con 0x7f4e24004850 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3d39a700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e24004850 0x7f4e24004cc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e28001940 tx=0x7f4e24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d260c4abd4a04601 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28001430 con 0x7f4e24004850 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28001d70 con 0x7f4e24004850 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.641+0000 7f4e3cb99700 1 -- 172.21.15.53:0/634246481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28005510 con 0x7f4e24004850 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 -- 172.21.15.53:0/634246481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e24004850 msgr2=0x7f4e24004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/634246481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e24004850 0x7f4e24004cc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e28001940 tx=0x7f4e24005880 comp rx=0 tx=0).stop 2023-11-28T11:20:45.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3e39c700 1 -- 172.21.15.53:0/634246481 reap_dead start 2023-11-28T11:20:45.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 -- 172.21.15.53:0/634246481 shutdown_connections 2023-11-28T11:20:45.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 -- 172.21.15.53:0/634246481 >> 172.21.15.53:0/634246481 conn(0x7f4e38072600 msgr2=0x7f4e380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:45.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 -- 172.21.15.53:0/634246481 shutdown_connections 2023-11-28T11:20:45.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.642+0000 7f4e3fdff700 1 -- 172.21.15.53:0/634246481 wait complete. 2023-11-28T11:20:45.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.644+0000 7f4e3fdff700 1 Processor -- start 2023-11-28T11:20:45.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.644+0000 7f4e3fdff700 1 -- start start 2023-11-28T11:20:45.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24004850 0x7f4e3814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e381507d0 0x7f4e381e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380cac20 con 0x7f4e380ca240 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e381f2620 con 0x7f4e24004850 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3fdff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e381e8db0 con 0x7f4e381507d0 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44092/0 (socket says 172.21.15.53:44092) 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 -- 172.21.15.53:0/2562135903 learned_addr learned my addr 172.21.15.53:0/2562135903 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3d39a700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e381507d0 0x7f4e381e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24004850 0x7f4e3814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 -- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e381507d0 msgr2=0x7f4e381e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e381507d0 0x7f4e381e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 -- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24004850 msgr2=0x7f4e3814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24004850 0x7f4e3814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.645+0000 7f4e3e39c700 1 -- 172.21.15.53:0/2562135903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380d15e0 con 0x7f4e380ca240 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e3e39c700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e30002230 tx=0x7f4e3000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000aba0 con 0x7f4e380ca240 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3000ad20 con 0x7f4e380ca240 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e380d1300 con 0x7f4e380ca240 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30013630 con 0x7f4e380ca240 2023-11-28T11:20:45.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.646+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3815cd80 con 0x7f4e380ca240 2023-11-28T11:20:45.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.647+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e380745c0 con 0x7f4e380ca240 2023-11-28T11:20:45.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.648+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e30013790 con 0x7f4e380ca240 2023-11-28T11:20:45.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.649+0000 7f4e2e7fc700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 0x7f4e1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:45.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.649+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4e300863d0 con 0x7f4e380ca240 2023-11-28T11:20:45.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.649+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 0x7f4e1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:45.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.650+0000 7f4e3db9b700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 0x7f4e1c06a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4e381e9070 tx=0x7f4e34009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:45.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.653+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e30052e30 con 0x7f4e380ca240 2023-11-28T11:20:45.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.858+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4e380d08a0 con 0x7f4e380ca240 2023-11-28T11:20:45.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.858+0000 7f4e2e7fc700 1 -- 172.21.15.53:0/2562135903 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4e30056450 con 0x7f4e380ca240 2023-11-28T11:20:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:45.861 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 2, 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "rbd" 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:45.862 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-28T11:20:45.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 msgr2=0x7f4e1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 0x7f4e1c06a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4e381e9070 tx=0x7f4e34009210 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 msgr2=0x7f4e380ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e30002230 tx=0x7f4e3000a510 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 shutdown_connections 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f4e1c0682f0 0x7f4e1c06a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e380ca240 0x7f4e380ca6e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e24004850 0x7f4e3814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 --2- 172.21.15.53:0/2562135903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e381507d0 0x7f4e381e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 >> 172.21.15.53:0/2562135903 conn(0x7f4e38072600 msgr2=0x7f4e3806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 shutdown_connections 2023-11-28T11:20:45.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:45.861+0000 7f4e3fdff700 1 -- 172.21.15.53:0/2562135903 wait complete. 2023-11-28T11:20:45.882 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:20:45.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:20:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.088+0000 7f7a70bff700 1 Processor -- start 2023-11-28T11:20:46.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.088+0000 7f7a70bff700 1 -- start start 2023-11-28T11:20:46.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a70bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a6c0cded0 0x7f7a6c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a70bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c160d70 con 0x7f7a6c15a760 2023-11-28T11:20:46.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a70bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c0b5390 con 0x7f7a6c15dc30 2023-11-28T11:20:46.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a70bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6c15e0d0 con 0x7f7a6c0cded0 2023-11-28T11:20:46.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a6a59c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7a6c15a760 0x7f7a6c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33890/0 (socket says 172.21.15.53:33890) 2023-11-28T11:20:46.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a6a59c700 1 -- 172.21.15.53:0/3602833764 learned_addr learned my addr 172.21.15.53:0/3602833764 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a63fff700 1 --2- 172.21.15.53:0/3602833764 >> v2:172.21.15.157:3300/0 conn(0x7f7a6c0cded0 0x7f7a6c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981525880 0 0) 0x7f7a6c160d70 con 0x7f7a6c15a760 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.090+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a580035b0 con 0x7f7a6c15a760 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3847226981 0 0) 0x7f7a580035b0 con 0x7f7a6c15a760 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 >> v2:172.21.15.157:3300/0 conn(0x7f7a6c0cded0 msgr2=0x7f7a6c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 --2- 172.21.15.53:0/3602833764 >> v2:172.21.15.157:3300/0 conn(0x7f7a6c0cded0 0x7f7a6c161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 >> v1:172.21.15.157:6789/0 conn(0x7f7a6c15dc30 legacy=0x7f7a6c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0b5670 con 0x7f7a6c15a760 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f7a5c001670 con 0x7f7a6c15a760 2023-11-28T11:20:46.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 >> v1:172.21.15.53:6789/0 conn(0x7f7a6c15a760 legacy=0x7f7a6c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a58003a50 0x7f7a58003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a58004850 0x7f7a58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c15e0d0 con 0x7f7a58005250 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5800c160 con 0x7f7a58003a50 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5800c320 con 0x7f7a58004850 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6a59c700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a58003a50 0x7f7a58003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a63fff700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a58004850 0x7f7a58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 -- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a58004850 msgr2=0x7f7a58004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a58004850 0x7f7a58004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 -- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a58003a50 msgr2=0x7f7a58003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a58003a50 0x7f7a58003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6ad9d700 1 -- 172.21.15.53:0/3602833764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580035b0 con 0x7f7a58005250 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.091+0000 7f7a6a59c700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a58003a50 0x7f7a58003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a6ad9d700 1 -- 172.21.15.53:0/3602833764 reap_dead start 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a6ad9d700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a5800ba10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c190 tx=0x7f7a58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8909571d6475376 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64001db0 con 0x7f7a58005250 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6400ccd0 con 0x7f7a58005250 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a69d9b700 1 -- 172.21.15.53:0/3602833764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64003c40 con 0x7f7a58005250 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a70bff700 1 -- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 msgr2=0x7f7a5800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.092+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a5800ba10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c190 tx=0x7f7a58005880 comp rx=0 tx=0).stop 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.093+0000 7f7a70bff700 1 -- 172.21.15.53:0/3602833764 shutdown_connections 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.093+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3602833764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a5800ba10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.093+0000 7f7a70bff700 1 -- 172.21.15.53:0/3602833764 >> 172.21.15.53:0/3602833764 conn(0x7f7a6c072620 msgr2=0x7f7a6c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.093+0000 7f7a70bff700 1 -- 172.21.15.53:0/3602833764 shutdown_connections 2023-11-28T11:20:46.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.093+0000 7f7a70bff700 1 -- 172.21.15.53:0/3602833764 wait complete. 2023-11-28T11:20:46.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.094+0000 7f7a70bff700 1 Processor -- start 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.095+0000 7f7a70bff700 1 -- start start 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.095+0000 7f7a70bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.095+0000 7f7a70bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6c0b7570 0x7f7a6c1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a70bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0b8b10 0x7f7a6c15f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a70bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0b8680 con 0x7f7a58005250 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a70bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0b7a70 con 0x7f7a6c0b8b10 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a70bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0b7d50 con 0x7f7a6c0b7570 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44112/0 (socket says 172.21.15.53:44112) 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 -- 172.21.15.53:0/3657734906 learned_addr learned my addr 172.21.15.53:0/3657734906 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a63fff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6c0b7570 0x7f7a6c1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6ad9d700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0b8b10 0x7f7a6c15f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 -- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6c0b7570 msgr2=0x7f7a6c1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6c0b7570 0x7f7a6c1569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 -- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0b8b10 msgr2=0x7f7a6c15f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0b8b10 0x7f7a6c15f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 -- 172.21.15.53:0/3657734906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c160a00 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.096+0000 7f7a6a59c700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e800 tx=0x7f7a5c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.097+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c012590 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.097+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c0126f0 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.097+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c012a60 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.097+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c160720 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.097+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c161200 con 0x7f7a58005250 2023-11-28T11:20:46.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.098+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c0b8960 con 0x7f7a58005250 2023-11-28T11:20:46.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.099+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a5c012bc0 con 0x7f7a58005250 2023-11-28T11:20:46.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.099+0000 7f7a62ffd700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 0x7f7a4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:46.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.100+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7a5c085800 con 0x7f7a58005250 2023-11-28T11:20:46.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.100+0000 7f7a63fff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 0x7f7a4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:46.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.101+0000 7f7a63fff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 0x7f7a4c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c15ca10 tx=0x7f7a54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:46.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.104+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a5c0523b0 con 0x7f7a58005250 2023-11-28T11:20:46.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.307+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7a6c04e620 con 0x7f7a58005250 2023-11-28T11:20:46.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:46.996+0000 7f7a62ffd700 1 -- 172.21.15.53:0/3657734906 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a5c001b30 con 0x7f7a58005250 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 msgr2=0x7f7a4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 0x7f7a4c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c15ca10 tx=0x7f7a54008040 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 msgr2=0x7f7a6c1e7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e800 tx=0x7f7a5c00ebc0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 shutdown_connections 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7a4c0682f0 0x7f7a4c06a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0b8b10 0x7f7a6c15f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a58005250 0x7f7a6c1e7980 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 --2- 172.21.15.53:0/3657734906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6c0b7570 0x7f7a6c1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.000+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 >> 172.21.15.53:0/3657734906 conn(0x7f7a6c072620 msgr2=0x7f7a6c158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.001+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 shutdown_connections 2023-11-28T11:20:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.001+0000 7f7a70bff700 1 -- 172.21.15.53:0/3657734906 wait complete. 2023-11-28T11:20:47.005 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-11-28T11:20:47.027 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:20:47.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.239+0000 7f3f6adff700 1 Processor -- start 2023-11-28T11:20:47.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.239+0000 7f3f6adff700 1 -- start start 2023-11-28T11:20:47.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.240+0000 7f3f6adff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f64159860 0x7f3f640c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.240+0000 7f3f6adff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640ccc00 con 0x7f3f64159490 2023-11-28T11:20:47.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.240+0000 7f3f6adff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640ccec0 con 0x7f3f640b0d80 2023-11-28T11:20:47.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.240+0000 7f3f6adff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640cd1a0 con 0x7f3f64159860 2023-11-28T11:20:47.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f68b9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3f64159490 0x7f3f640b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33896/0 (socket says 172.21.15.53:33896) 2023-11-28T11:20:47.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f68b9b700 1 -- 172.21.15.53:0/2255478042 learned_addr learned my addr 172.21.15.53:0/2255478042 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:47.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f64159860 0x7f3f640c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837270593 0 0) 0x7f3f640ccc00 con 0x7f3f64159490 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f540035b0 con 0x7f3f64159490 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223000451 0 0) 0x7f3f640ccec0 con 0x7f3f640b0d80 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f640ccc00 con 0x7f3f640b0d80 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3712382308 0 0) 0x7f3f540035b0 con 0x7f3f64159490 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f64159860 msgr2=0x7f3f640c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f64159860 0x7f3f640c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 >> v1:172.21.15.157:6789/0 conn(0x7f3f640b0d80 legacy=0x7f3f640c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640cd480 con 0x7f3f64159490 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f3f58001670 con 0x7f3f64159490 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.241+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 >> v1:172.21.15.53:6789/0 conn(0x7f3f64159490 legacy=0x7f3f640b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54003bc0 0x7f3f54003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f54004850 0x7f3f54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f54005200 0x7f3f5400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640cd1a0 con 0x7f3f54004850 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5400e160 con 0x7f3f54005200 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5400e2a0 con 0x7f3f54003bc0 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f54004850 0x7f3f54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f68b9b700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54003bc0 0x7f3f54003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f6939c700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f54005200 0x7f3f5400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 -- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54003bc0 msgr2=0x7f3f54003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54003bc0 0x7f3f54003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 -- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f54005200 msgr2=0x7f3f5400dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f54005200 0x7f3f5400dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f63fff700 1 -- 172.21.15.53:0/2255478042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540035b0 con 0x7f3f54004850 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.242+0000 7f3f6939c700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f54005200 0x7f3f5400dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:47.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f54004850 0x7f3f54004cc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009b90 tx=0x7f3f54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ebcaab26deb451 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000e6c0 con 0x7f3f54004850 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5000e820 con 0x7f3f54004850 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f637fe700 1 -- 172.21.15.53:0/2255478042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000eb90 con 0x7f3f54004850 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6adff700 1 -- 172.21.15.53:0/2255478042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f54004850 msgr2=0x7f3f54004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2255478042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f54004850 0x7f3f54004cc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009b90 tx=0x7f3f54005880 comp rx=0 tx=0).stop 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6939c700 1 -- 172.21.15.53:0/2255478042 reap_dead start 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6adff700 1 -- 172.21.15.53:0/2255478042 shutdown_connections 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6adff700 1 -- 172.21.15.53:0/2255478042 >> 172.21.15.53:0/2255478042 conn(0x7f3f640cb670 msgr2=0x7f3f640cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.243+0000 7f3f6adff700 1 -- 172.21.15.53:0/2255478042 shutdown_connections 2023-11-28T11:20:47.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.244+0000 7f3f6adff700 1 -- 172.21.15.53:0/2255478042 wait complete. 2023-11-28T11:20:47.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.246+0000 7f3f6adff700 1 Processor -- start 2023-11-28T11:20:47.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.246+0000 7f3f6adff700 1 -- start start 2023-11-28T11:20:47.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54004850 0x7f3f641531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 0x7f3f640b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64153730 con 0x7f3f6415ac40 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c86d0 con 0x7f3f6415b9b0 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.247+0000 7f3f6adff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6415be50 con 0x7f3f54004850 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44138/0 (socket says 172.21.15.53:44138) 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 -- 172.21.15.53:0/2877618280 learned_addr learned my addr 172.21.15.53:0/2877618280 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f68b9b700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54004850 0x7f3f641531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f6939c700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 0x7f3f640b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 -- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54004850 msgr2=0x7f3f641531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54004850 0x7f3f641531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 -- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 msgr2=0x7f3f640b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:47.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 0x7f3f640b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 -- 172.21.15.53:0/2877618280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640cdc80 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f6939c700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 0x7f3f640b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.248+0000 7f3f63fff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000e4b0 tx=0x7f3f500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.249+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50021070 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.249+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f50004490 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.249+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f640cd760 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.249+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000e6c0 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.249+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f640cde10 con 0x7f3f6415ac40 2023-11-28T11:20:47.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.250+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f640c5950 con 0x7f3f6415ac40 2023-11-28T11:20:47.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.251+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f5002e030 con 0x7f3f6415ac40 2023-11-28T11:20:47.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.252+0000 7f3f617fa700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 0x7f3f4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:47.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.252+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3f50086df0 con 0x7f3f6415ac40 2023-11-28T11:20:47.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.253+0000 7f3f68b9b700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 0x7f3f4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:47.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.253+0000 7f3f68b9b700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 0x7f3f4806a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f58001e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:47.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.257+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f50053740 con 0x7f3f6415ac40 2023-11-28T11:20:47.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.463+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f640745c0 con 0x7f3f6415ac40 2023-11-28T11:20:48.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:47.999+0000 7f3f617fa700 1 -- 172.21.15.53:0/2877618280 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3f50056d60 con 0x7f3f6415ac40 2023-11-28T11:20:48.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.005+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 msgr2=0x7f3f4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.005+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 0x7f3f4806a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f58001e20 comp rx=0 tx=0).stop 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.005+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 msgr2=0x7f3f6415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.005+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000e4b0 tx=0x7f3f500047c0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 shutdown_connections 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3f480682f0 0x7f3f4806a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f54004850 0x7f3f641531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f6415b9b0 0x7f3f640b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 --2- 172.21.15.53:0/2877618280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f6415ac40 0x7f3f6415b470 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.006+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 >> 172.21.15.53:0/2877618280 conn(0x7f3f640cb670 msgr2=0x7f3f64071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.007+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 shutdown_connections 2023-11-28T11:20:48.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.007+0000 7f3f6adff700 1 -- 172.21.15.53:0/2877618280 wait complete. 2023-11-28T11:20:48.015 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-11-28T11:20:48.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:20:48.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.241+0000 7f7d6d7ff700 1 Processor -- start 2023-11-28T11:20:48.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.241+0000 7f7d6d7ff700 1 -- start start 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d6d7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d6815c9b0 0x7f7d680b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d6d7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d68161340 0x7f7d680d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d6d7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680d1950 con 0x7f7d680d2320 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d6d7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680b5800 con 0x7f7d6815c9b0 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d6d7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680b5520 con 0x7f7d68161340 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d66ffd700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d6815c9b0 0x7f7d680b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d677fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d680d2320 0x7f7d680cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33910/0 (socket says 172.21.15.53:33910) 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.242+0000 7f7d677fe700 1 -- 172.21.15.53:0/566695222 learned_addr learned my addr 172.21.15.53:0/566695222 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d5ffff700 1 --2- 172.21.15.53:0/566695222 >> v2:172.21.15.157:3300/0 conn(0x7f7d68161340 0x7f7d680d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d66ffd700 1 -- 172.21.15.53:0/566695222 >> v2:172.21.15.157:3300/0 conn(0x7f7d68161340 msgr2=0x7f7d680d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d66ffd700 1 --2- 172.21.15.53:0/566695222 >> v2:172.21.15.157:3300/0 conn(0x7f7d68161340 0x7f7d680d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d66ffd700 1 -- 172.21.15.53:0/566695222 >> v1:172.21.15.53:6789/0 conn(0x7f7d680d2320 legacy=0x7f7d680cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d66ffd700 1 -- 172.21.15.53:0/566695222 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680d1300 con 0x7f7d6815c9b0 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d66ffd700 1 --2- 172.21.15.53:0/566695222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6815c9b0 0x7f7d680b4bf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7d680685c0 tx=0x7f7d58009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efa447fe566a49c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d667fc700 1 -- 172.21.15.53:0/566695222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58011830 con 0x7f7d6815c9b0 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d667fc700 1 -- 172.21.15.53:0/566695222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58011990 con 0x7f7d6815c9b0 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.243+0000 7f7d667fc700 1 -- 172.21.15.53:0/566695222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58011d20 con 0x7f7d6815c9b0 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/566695222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6815c9b0 msgr2=0x7f7d680b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/566695222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6815c9b0 0x7f7d680b4bf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7d680685c0 tx=0x7f7d58009b20 comp rx=0 tx=0).stop 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/566695222 shutdown_connections 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/566695222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6815c9b0 0x7f7d680b4bf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/566695222 >> v2:172.21.15.157:3300/0 conn(0x7f7d68161340 0x7f7d680d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/566695222 >> 172.21.15.53:0/566695222 conn(0x7f7d68072600 msgr2=0x7f7d680cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/566695222 shutdown_connections 2023-11-28T11:20:48.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.244+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/566695222 wait complete. 2023-11-28T11:20:48.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.247+0000 7f7d6d7ff700 1 Processor -- start 2023-11-28T11:20:48.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.247+0000 7f7d6d7ff700 1 -- start start 2023-11-28T11:20:48.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d6815c9b0 0x7f7d68155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d68161340 0x7f7d68157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68152e30 con 0x7f7d680d2320 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68153110 con 0x7f7d6815c9b0 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d6d7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681533f0 con 0x7f7d68161340 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44158/0 (socket says 172.21.15.53:44158) 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d66ffd700 1 -- 172.21.15.53:0/2610867144 learned_addr learned my addr 172.21.15.53:0/2610867144 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d5ffff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d6815c9b0 0x7f7d68155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.248+0000 7f7d677fe700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d68161340 0x7f7d68157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 -- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d68161340 msgr2=0x7f7d68157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d68161340 0x7f7d68157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 -- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d6815c9b0 msgr2=0x7f7d68155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d6815c9b0 0x7f7d68155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 -- 172.21.15.53:0/2610867144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580094a0 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d66ffd700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7d58000c00 tx=0x7f7d58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580036a0 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5801f070 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003a60 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d680b5130 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.249+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d680d15e0 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.250+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d680b52c0 con 0x7f7d680d2320 2023-11-28T11:20:48.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.252+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d5800f050 con 0x7f7d680d2320 2023-11-28T11:20:48.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.252+0000 7f7d5f7fe700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.252+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f7d58021070 con 0x7f7d680d2320 2023-11-28T11:20:48.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.253+0000 7f7d5ffff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.254+0000 7f7d5ffff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d6815f770 tx=0x7f7d50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.258+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d58053e50 con 0x7f7d680d2320 2023-11-28T11:20:48.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.506+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7d68152f70 con 0x7f7d680d2320 2023-11-28T11:20:48.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.512+0000 7f7d5f7fe700 1 -- 172.21.15.53:0/2610867144 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7d68152f70 con 0x7f7d680d2320 2023-11-28T11:20:48.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.517+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 msgr2=0x7f7d4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.517+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d6815f770 tx=0x7f7d50008040 comp rx=0 tx=0).stop 2023-11-28T11:20:48.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.517+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 msgr2=0x7f7d681f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.517+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7d58000c00 tx=0x7f7d58004750 comp rx=0 tx=0).stop 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 shutdown_connections 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d68161340 0x7f7d68157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d680d2320 0x7f7d681f2a10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 --2- 172.21.15.53:0/2610867144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d6815c9b0 0x7f7d68155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.520+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 >> 172.21.15.53:0/2610867144 conn(0x7f7d68072600 msgr2=0x7f7d6815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.521+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 shutdown_connections 2023-11-28T11:20:48.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.521+0000 7f7d6d7ff700 1 -- 172.21.15.53:0/2610867144 wait complete. 2023-11-28T11:20:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T11:20:48.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:48.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.758+0000 7f52463ff700 1 Processor -- start 2023-11-28T11:20:48.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.759+0000 7f52463ff700 1 -- start start 2023-11-28T11:20:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f52463ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f524015c9b0 0x7f52400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f52463ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5240161340 0x7f52400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f52463ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400d1950 con 0x7f52400d2320 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f52463ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52400b5800 con 0x7f524015c9b0 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f52463ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52400b5520 con 0x7f5240161340 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f523ffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f524015c9b0 0x7f52400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f524499c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f52400d2320 0x7f52400cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33922/0 (socket says 172.21.15.53:33922) 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f524499c700 1 -- 172.21.15.53:0/2747516066 learned_addr learned my addr 172.21.15.53:0/2747516066 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f523f7fe700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5240161340 0x7f52400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f523ffff700 1 -- 172.21.15.53:0/2747516066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5240161340 msgr2=0x7f52400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f523ffff700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5240161340 0x7f52400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.760+0000 7f523ffff700 1 -- 172.21.15.53:0/2747516066 >> v1:172.21.15.53:6789/0 conn(0x7f52400d2320 legacy=0x7f52400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523ffff700 1 -- 172.21.15.53:0/2747516066 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400d1300 con 0x7f524015c9b0 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523effd700 1 -- 172.21.15.53:0/2747516066 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210122800 0 0) 0x7f52400d1950 con 0x7f52400d2320 2023-11-28T11:20:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523ffff700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524015c9b0 0x7f52400b4bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5230009840 tx=0x7f5230009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ea69c911f9a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523effd700 1 -- 172.21.15.53:0/2747516066 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230011830 con 0x7f524015c9b0 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523effd700 1 -- 172.21.15.53:0/2747516066 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230011990 con 0x7f524015c9b0 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.761+0000 7f523effd700 1 -- 172.21.15.53:0/2747516066 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230011d20 con 0x7f524015c9b0 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 -- 172.21.15.53:0/2747516066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524015c9b0 msgr2=0x7f52400b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524015c9b0 0x7f52400b4bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5230009840 tx=0x7f5230009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 -- 172.21.15.53:0/2747516066 shutdown_connections 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524015c9b0 0x7f52400b4bf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 --2- 172.21.15.53:0/2747516066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5240161340 0x7f52400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 -- 172.21.15.53:0/2747516066 >> 172.21.15.53:0/2747516066 conn(0x7f5240072600 msgr2=0x7f52400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 -- 172.21.15.53:0/2747516066 shutdown_connections 2023-11-28T11:20:48.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.762+0000 7f52463ff700 1 -- 172.21.15.53:0/2747516066 wait complete. 2023-11-28T11:20:48.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.763+0000 7f52463ff700 1 Processor -- start 2023-11-28T11:20:48.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.764+0000 7f52463ff700 1 -- start start 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.764+0000 7f52463ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52400d2320 0x7f52401540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.764+0000 7f52463ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524015c9b0 0x7f5240152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f52463ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f52463ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240153cd0 con 0x7f5240161340 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f52463ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240150b20 con 0x7f524015c9b0 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f52463ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5240153050 con 0x7f52400d2320 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44170/0 (socket says 172.21.15.53:44170) 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 -- 172.21.15.53:0/349689200 learned_addr learned my addr 172.21.15.53:0/349689200 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f523f7fe700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524015c9b0 0x7f5240152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f523ffff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52400d2320 0x7f52401540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 -- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52400d2320 msgr2=0x7f52401540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52400d2320 0x7f52401540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 -- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524015c9b0 msgr2=0x7f5240152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524015c9b0 0x7f5240152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.765+0000 7f524499c700 1 -- 172.21.15.53:0/349689200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300094a0 con 0x7f5240161340 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f524499c700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f523800cb60 tx=0x7f523800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800ab50 con 0x7f5240161340 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523800acb0 con 0x7f5240161340 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400b5130 con 0x7f5240161340 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238007310 con 0x7f5240161340 2023-11-28T11:20:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.766+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52400d15e0 con 0x7f5240161340 2023-11-28T11:20:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.767+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52400b52c0 con 0x7f5240161340 2023-11-28T11:20:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.768+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5238010070 con 0x7f5240161340 2023-11-28T11:20:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.769+0000 7f523cff9700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 0x7f522c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.769+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52380857d0 con 0x7f5240161340 2023-11-28T11:20:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.769+0000 7f523ffff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 0x7f522c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.770+0000 7f523ffff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 0x7f522c06a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52300090b0 tx=0x7f523001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.773+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5238051f40 con 0x7f5240161340 2023-11-28T11:20:48.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.975+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5240153e10 con 0x7f5240161340 2023-11-28T11:20:48.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.977+0000 7f523cff9700 1 -- 172.21.15.53:0/349689200 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17934 (secure 0 0 0) 0x7f5240153e10 con 0x7f5240161340 2023-11-28T11:20:48.979 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:48.979 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:20:48.508267+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 msgr2=0x7f522c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 0x7f522c06a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52300090b0 tx=0x7f523001c040 comp rx=0 tx=0).stop 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 msgr2=0x7f52401505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f523800cb60 tx=0x7f523800cf20 comp rx=0 tx=0).stop 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 shutdown_connections 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f522c0682f0 0x7f522c06a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52400d2320 0x7f52401540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5240161340 0x7f52401505e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 --2- 172.21.15.53:0/349689200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f524015c9b0 0x7f5240152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 >> 172.21.15.53:0/349689200 conn(0x7f5240072600 msgr2=0x7f524015cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 shutdown_connections 2023-11-28T11:20:48.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:48.979+0000 7f52463ff700 1 -- 172.21.15.53:0/349689200 wait complete. 2023-11-28T11:20:49.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.199+0000 7f77aabff700 1 Processor -- start 2023-11-28T11:20:49.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.200+0000 7f77aabff700 1 -- start start 2023-11-28T11:20:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77aabff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77a4161340 0x7f77a40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77aabff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77a40d2320 0x7f77a40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77aabff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40d1950 con 0x7f77a415c9b0 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77aabff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40b5800 con 0x7f77a4161340 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77aabff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40b5300 con 0x7f77a40d2320 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77a3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77a4161340 0x7f77a40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77a3fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77a4161340 0x7f77a40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44186/0 (socket says 172.21.15.53:44186) 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77a3fff700 1 -- 172.21.15.53:0/3874873086 learned_addr learned my addr 172.21.15.53:0/3874873086 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.201+0000 7f77a919c700 1 --2- 172.21.15.53:0/3874873086 >> v2:172.21.15.157:3300/0 conn(0x7f77a40d2320 0x7f77a40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a3fff700 1 -- 172.21.15.53:0/3874873086 >> v2:172.21.15.157:3300/0 conn(0x7f77a40d2320 msgr2=0x7f77a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:49.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a3fff700 1 --2- 172.21.15.53:0/3874873086 >> v2:172.21.15.157:3300/0 conn(0x7f77a40d2320 0x7f77a40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a3fff700 1 -- 172.21.15.53:0/3874873086 >> v1:172.21.15.157:6789/0 conn(0x7f77a415c9b0 legacy=0x7f77a40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a3fff700 1 -- 172.21.15.53:0/3874873086 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40b5520 con 0x7f77a4161340 2023-11-28T11:20:49.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a3fff700 1 --2- 172.21.15.53:0/3874873086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a40d1de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77a406c170 tx=0x7f7790009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef132534e796a7b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a37fe700 1 -- 172.21.15.53:0/3874873086 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000b860 con 0x7f77a4161340 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a37fe700 1 -- 172.21.15.53:0/3874873086 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779000b9c0 con 0x7f77a4161340 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.202+0000 7f77a37fe700 1 -- 172.21.15.53:0/3874873086 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000bd50 con 0x7f77a4161340 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 -- 172.21.15.53:0/3874873086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 msgr2=0x7f77a40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 --2- 172.21.15.53:0/3874873086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a40d1de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77a406c170 tx=0x7f7790009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 -- 172.21.15.53:0/3874873086 shutdown_connections 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 --2- 172.21.15.53:0/3874873086 >> v2:172.21.15.157:3300/0 conn(0x7f77a40d2320 0x7f77a40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 --2- 172.21.15.53:0/3874873086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a40d1de0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 -- 172.21.15.53:0/3874873086 >> 172.21.15.53:0/3874873086 conn(0x7f77a4072600 msgr2=0x7f77a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:49.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 -- 172.21.15.53:0/3874873086 shutdown_connections 2023-11-28T11:20:49.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.203+0000 7f77aabff700 1 -- 172.21.15.53:0/3874873086 wait complete. 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.205+0000 7f77aabff700 1 Processor -- start 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.205+0000 7f77aabff700 1 -- start start 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40d2320 0x7f77a4150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a415c9b0 0x7f77a41e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41547e0 con 0x7f77a4161340 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41e6460 con 0x7f77a40d2320 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77aabff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41e72c0 con 0x7f77a415c9b0 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77a919c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77a919c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44192/0 (socket says 172.21.15.53:44192) 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77a919c700 1 -- 172.21.15.53:0/3365214516 learned_addr learned my addr 172.21.15.53:0/3365214516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77a899b700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40d2320 0x7f77a4150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.206+0000 7f77a3fff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a415c9b0 0x7f77a41e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 -- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a415c9b0 msgr2=0x7f77a41e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a415c9b0 0x7f77a41e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 -- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40d2320 msgr2=0x7f77a4150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40d2320 0x7f77a4150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 -- 172.21.15.53:0/3365214516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790009550 con 0x7f77a4161340 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a919c700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f779c007d80 tx=0x7f779c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00ed30 con 0x7f77a4161340 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00ee90 con 0x7f77a4161340 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a40d15e0 con 0x7f77a4161340 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c014660 con 0x7f77a4161340 2023-11-28T11:20:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.207+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a40d1300 con 0x7f77a4161340 2023-11-28T11:20:49.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.208+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a415cd80 con 0x7f77a4161340 2023-11-28T11:20:49.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.209+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f779c012050 con 0x7f77a4161340 2023-11-28T11:20:49.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.211+0000 7f77a17fa700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.211+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f779c0861c0 con 0x7f77a4161340 2023-11-28T11:20:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.212+0000 7f77a899b700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.212+0000 7f77a899b700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 0x7f778806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7798007fd0 tx=0x7f7798009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.215+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f779c052930 con 0x7f77a4161340 2023-11-28T11:20:49.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.420+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a415d0c0 con 0x7f77a4161340 2023-11-28T11:20:49.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.421+0000 7f77a17fa700 1 -- 172.21.15.53:0/3365214516 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17934 (secure 0 0 0) 0x7f779c055f50 con 0x7f77a4161340 2023-11-28T11:20:49.424 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:49.424 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:20:48.508267+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 msgr2=0x7f778806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 0x7f778806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7798007fd0 tx=0x7f7798009040 comp rx=0 tx=0).stop 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 msgr2=0x7f77a4156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f779c007d80 tx=0x7f779c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 shutdown_connections 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a4161340 0x7f77a4156230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40d2320 0x7f77a4150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 --2- 172.21.15.53:0/3365214516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a415c9b0 0x7f77a41e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 >> 172.21.15.53:0/3365214516 conn(0x7f77a4072600 msgr2=0x7f77a40b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 shutdown_connections 2023-11-28T11:20:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.424+0000 7f77aabff700 1 -- 172.21.15.53:0/3365214516 wait complete. 2023-11-28T11:20:49.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:20:49.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.649+0000 7fb600dff700 1 Processor -- start 2023-11-28T11:20:49.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.649+0000 7fb600dff700 1 -- start start 2023-11-28T11:20:49.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.650+0000 7fb600dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc15c9b0 0x7fb5fc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.650+0000 7fb600dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb600dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc0ce160 con 0x7fb5fc161340 2023-11-28T11:20:49.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb600dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b5800 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb600dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0d1950 con 0x7fb5fc15c9b0 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44206/0 (socket says 172.21.15.53:44206) 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 -- 172.21.15.53:0/3941766399 learned_addr learned my addr 172.21.15.53:0/3941766399 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fa59c700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc15c9b0 0x7fb5fc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 -- 172.21.15.53:0/3941766399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc15c9b0 msgr2=0x7fb5fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc15c9b0 0x7fb5fc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 -- 172.21.15.53:0/3941766399 >> v1:172.21.15.157:6789/0 conn(0x7fb5fc161340 legacy=0x7fb5fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.651+0000 7fb5fad9d700 1 -- 172.21.15.53:0/3941766399 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0d24f0 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb5fad9d700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009e50 tx=0x7fb5f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13900237d8c3a1e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb5f9d9b700 1 -- 172.21.15.53:0/3941766399 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400e070 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb5f9d9b700 1 -- 172.21.15.53:0/3941766399 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400b9c0 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb5f9d9b700 1 -- 172.21.15.53:0/3941766399 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400bd70 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb600dff700 1 -- 172.21.15.53:0/3941766399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc0cd7c0 msgr2=0x7fb5fc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.652+0000 7fb600dff700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009e50 tx=0x7fb5f400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 -- 172.21.15.53:0/3941766399 shutdown_connections 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc0cd7c0 0x7fb5fc0cdc20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 --2- 172.21.15.53:0/3941766399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc15c9b0 0x7fb5fc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 -- 172.21.15.53:0/3941766399 >> 172.21.15.53:0/3941766399 conn(0x7fb5fc072600 msgr2=0x7fb5fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 -- 172.21.15.53:0/3941766399 shutdown_connections 2023-11-28T11:20:49.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.653+0000 7fb600dff700 1 -- 172.21.15.53:0/3941766399 wait complete. 2023-11-28T11:20:49.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.655+0000 7fb600dff700 1 Processor -- start 2023-11-28T11:20:49.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.655+0000 7fb600dff700 1 -- start start 2023-11-28T11:20:49.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.656+0000 7fb600dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 0x7fb5fc154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.656+0000 7fb600dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 0x7fb5fc1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb600dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb600dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc151b30 con 0x7fb5fc161340 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb600dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1569e0 con 0x7fb5fc15c9b0 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb600dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc156cc0 con 0x7fb5fc0cd7c0 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44208/0 (socket says 172.21.15.53:44208) 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 -- 172.21.15.53:0/4139995836 learned_addr learned my addr 172.21.15.53:0/4139995836 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 -- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 msgr2=0x7fb5fc154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fa59c700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 0x7fb5fc154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5f3fff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 0x7fb5fc1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 0x7fb5fc154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 -- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 msgr2=0x7fb5fc1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 0x7fb5fc1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fad9d700 1 -- 172.21.15.53:0/4139995836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009b30 con 0x7fb5fc161340 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5fa59c700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 0x7fb5fc154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.657+0000 7fb5f3fff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 0x7fb5fc1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb5fad9d700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc067550 tx=0x7fb5f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004960 con 0x7fb5fc161340 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f40099b0 con 0x7fb5fc161340 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f401b670 con 0x7fb5fc161340 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc0d1300 con 0x7fb5fc161340 2023-11-28T11:20:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.658+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc0b5520 con 0x7fb5fc161340 2023-11-28T11:20:49.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.659+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc0d15e0 con 0x7fb5fc161340 2023-11-28T11:20:49.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.660+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5f402b070 con 0x7fb5fc161340 2023-11-28T11:20:49.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.661+0000 7fb5f2ffd700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.661+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5f4087610 con 0x7fb5fc161340 2023-11-28T11:20:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.662+0000 7fb5fa59c700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.663+0000 7fb5fa59c700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009820 tx=0x7fb5ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.666+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5f4053d80 con 0x7fb5fc161340 2023-11-28T11:20:49.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:49.922+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5fc0745c0 con 0x7fb5fc161340 2023-11-28T11:20:50.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.523+0000 7fb5f2ffd700 1 -- 172.21.15.53:0/4139995836 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb5f40573a0 con 0x7fb5fc161340 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 msgr2=0x7fb5dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009820 tx=0x7fb5ec008040 comp rx=0 tx=0).stop 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 msgr2=0x7fb5fc15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc067550 tx=0x7fb5f4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 shutdown_connections 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5fc0cd7c0 0x7fb5fc154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5fc161340 0x7fb5fc15eaf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 --2- 172.21.15.53:0/4139995836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5fc15c9b0 0x7fb5fc1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 >> 172.21.15.53:0/4139995836 conn(0x7fb5fc072600 msgr2=0x7fb5fc0b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:50.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 shutdown_connections 2023-11-28T11:20:50.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.526+0000 7fb600dff700 1 -- 172.21.15.53:0/4139995836 wait complete. 2023-11-28T11:20:50.549 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:20:50.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:20:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.755+0000 7fb271fff700 1 Processor -- start 2023-11-28T11:20:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.755+0000 7fb271fff700 1 -- start start 2023-11-28T11:20:50.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.756+0000 7fb271fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.756+0000 7fb271fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb26c0cd7c0 0x7fb26c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.756+0000 7fb271fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0ce160 con 0x7fb26c161340 2023-11-28T11:20:50.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.756+0000 7fb271fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c0b5800 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.756+0000 7fb271fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0d1950 con 0x7fb26c15c9b0 2023-11-28T11:20:50.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:36258/0 (socket says 172.21.15.53:36258) 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 -- 172.21.15.53:0/539779131 learned_addr learned my addr 172.21.15.53:0/539779131 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26bfff700 1 --2- 172.21.15.53:0/539779131 >> v2:172.21.15.157:3300/0 conn(0x7fb26c0cd7c0 0x7fb26c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26affd700 1 --1- 172.21.15.53:0/539779131 >> v1:172.21.15.157:6789/0 conn(0x7fb26c161340 0x7fb26c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:38832/0 (socket says 172.21.15.53:38832) 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 -- 172.21.15.53:0/539779131 >> v2:172.21.15.157:3300/0 conn(0x7fb26c0cd7c0 msgr2=0x7fb26c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 --2- 172.21.15.53:0/539779131 >> v2:172.21.15.157:3300/0 conn(0x7fb26c0cd7c0 0x7fb26c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 -- 172.21.15.53:0/539779131 >> v1:172.21.15.157:6789/0 conn(0x7fb26c161340 legacy=0x7fb26c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26b7fe700 1 -- 172.21.15.53:0/539779131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0d24f0 con 0x7fb26c15c9b0 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.757+0000 7fb26a7fc700 1 -- 172.21.15.53:0/539779131 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227043810 0 0) 0x7fb26c0ce160 con 0x7fb26c161340 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb26b7fe700 1 --2- 172.21.15.53:0/539779131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0098f0 tx=0x7fb25c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66434eb000e5535 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb26a7fc700 1 -- 172.21.15.53:0/539779131 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011830 con 0x7fb26c15c9b0 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb26a7fc700 1 -- 172.21.15.53:0/539779131 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c011990 con 0x7fb26c15c9b0 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb26a7fc700 1 -- 172.21.15.53:0/539779131 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011d20 con 0x7fb26c15c9b0 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 -- 172.21.15.53:0/539779131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 msgr2=0x7fb26c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 --2- 172.21.15.53:0/539779131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0098f0 tx=0x7fb25c009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 -- 172.21.15.53:0/539779131 shutdown_connections 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 --2- 172.21.15.53:0/539779131 >> v2:172.21.15.157:3300/0 conn(0x7fb26c0cd7c0 0x7fb26c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 --2- 172.21.15.53:0/539779131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c0b4bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 -- 172.21.15.53:0/539779131 >> 172.21.15.53:0/539779131 conn(0x7fb26c072600 msgr2=0x7fb26c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.758+0000 7fb271fff700 1 -- 172.21.15.53:0/539779131 shutdown_connections 2023-11-28T11:20:50.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.759+0000 7fb271fff700 1 -- 172.21.15.53:0/539779131 wait complete. 2023-11-28T11:20:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.760+0000 7fb271fff700 1 Processor -- start 2023-11-28T11:20:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.760+0000 7fb271fff700 1 -- start start 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c161340 0x7fb26c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c14dd10 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c158ce0 con 0x7fb26c161340 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb271fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c158fc0 con 0x7fb26c15c9b0 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.761+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44222/0 (socket says 172.21.15.53:44222) 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 -- 172.21.15.53:0/3612495725 learned_addr learned my addr 172.21.15.53:0/3612495725 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26bfff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c161340 0x7fb26c0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26affd700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 -- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 msgr2=0x7fb26c14d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c14d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 -- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c161340 msgr2=0x7fb26c0c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c161340 0x7fb26c0c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 -- 172.21.15.53:0/3612495725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c009550 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.762+0000 7fb26b7fe700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009c40 tx=0x7fb25c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.763+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004510 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.763+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c01e070 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.763+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0038c0 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.763+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c0d1300 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.763+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c0b5520 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.764+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c0d15e0 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.765+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb25c00f050 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.766+0000 7fb253fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 0x7fb25806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.766+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb25c086bd0 con 0x7fb26c0cd7c0 2023-11-28T11:20:50.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.766+0000 7fb26affd700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 0x7fb25806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.767+0000 7fb26affd700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 0x7fb25806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b5450 tx=0x7fb260007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:50.770+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb25c0532c0 con 0x7fb26c0cd7c0 2023-11-28T11:20:51.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.021+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb26c0745c0 con 0x7fb26c0cd7c0 2023-11-28T11:20:51.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.525+0000 7fb253fff700 1 -- 172.21.15.53:0/3612495725 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb25c0200c0 con 0x7fb26c0cd7c0 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 msgr2=0x7fb25806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 0x7fb25806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b5450 tx=0x7fb260007e70 comp rx=0 tx=0).stop 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 msgr2=0x7fb26c159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009c40 tx=0x7fb25c004750 comp rx=0 tx=0).stop 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 shutdown_connections 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb2580682f0 0x7fb25806a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb26c161340 0x7fb26c0c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26c0cd7c0 0x7fb26c159d80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 --2- 172.21.15.53:0/3612495725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb26c15c9b0 0x7fb26c14d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.528+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 >> 172.21.15.53:0/3612495725 conn(0x7fb26c072600 msgr2=0x7fb26c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.529+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 shutdown_connections 2023-11-28T11:20:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.529+0000 7fb271fff700 1 -- 172.21.15.53:0/3612495725 wait complete. 2023-11-28T11:20:51.552 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:20:51.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:51.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.754+0000 7f95b21ff700 1 Processor -- start 2023-11-28T11:20:51.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.754+0000 7f95b21ff700 1 -- start start 2023-11-28T11:20:51.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.755+0000 7f95b21ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95ac161340 0x7f95ac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.755+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac0d2320 0x7f95ac0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.755+0000 7f95b21ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0d1950 con 0x7f95ac15c9b0 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.755+0000 7f95b21ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0b5800 con 0x7f95ac161340 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.755+0000 7f95b21ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0b5300 con 0x7f95ac0d2320 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95ab7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f95ac15c9b0 0x7f95ac0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36268/0 (socket says 172.21.15.53:36268) 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95ab7fe700 1 -- 172.21.15.53:0/2302464458 learned_addr learned my addr 172.21.15.53:0/2302464458 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:51.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 --2- 172.21.15.53:0/2302464458 >> v2:172.21.15.53:3300/0 conn(0x7f95ac161340 0x7f95ac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95abfff700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac0d2320 0x7f95ac0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 -- 172.21.15.53:0/2302464458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac0d2320 msgr2=0x7f95ac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac0d2320 0x7f95ac0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 -- 172.21.15.53:0/2302464458 >> v1:172.21.15.53:6789/0 conn(0x7f95ac15c9b0 legacy=0x7f95ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 -- 172.21.15.53:0/2302464458 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0b5520 con 0x7f95ac161340 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.756+0000 7f95aaffd700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac161340 0x7f95ac0d1de0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009840 tx=0x7f95a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8102298d6628ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95aa7fc700 1 -- 172.21.15.53:0/2302464458 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011830 con 0x7f95ac161340 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95aa7fc700 1 -- 172.21.15.53:0/2302464458 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0011990 con 0x7f95ac161340 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95aa7fc700 1 -- 172.21.15.53:0/2302464458 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011d20 con 0x7f95ac161340 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 -- 172.21.15.53:0/2302464458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac161340 msgr2=0x7f95ac0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac161340 0x7f95ac0d1de0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009840 tx=0x7f95a0009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 -- 172.21.15.53:0/2302464458 shutdown_connections 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac0d2320 0x7f95ac0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 --2- 172.21.15.53:0/2302464458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac161340 0x7f95ac0d1de0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 -- 172.21.15.53:0/2302464458 >> 172.21.15.53:0/2302464458 conn(0x7f95ac072600 msgr2=0x7f95ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 -- 172.21.15.53:0/2302464458 shutdown_connections 2023-11-28T11:20:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.757+0000 7f95b21ff700 1 -- 172.21.15.53:0/2302464458 wait complete. 2023-11-28T11:20:51.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.759+0000 7f95b21ff700 1 Processor -- start 2023-11-28T11:20:51.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.759+0000 7f95b21ff700 1 -- start start 2023-11-28T11:20:51.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac15c9b0 0x7f95ac1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac161340 0x7f95ac1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1509d0 con 0x7f95ac0d2320 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1e8920 con 0x7f95ac161340 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95b21ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1e79c0 con 0x7f95ac15c9b0 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38146/0 (socket says 172.21.15.53:38146) 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.760+0000 7f95ab7fe700 1 -- 172.21.15.53:0/254459439 learned_addr learned my addr 172.21.15.53:0/254459439 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95abfff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac161340 0x7f95ac1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95aaffd700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac15c9b0 0x7f95ac1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 -- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac15c9b0 msgr2=0x7f95ac1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac15c9b0 0x7f95ac1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 -- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac161340 msgr2=0x7f95ac1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac161340 0x7f95ac1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 -- 172.21.15.53:0/254459439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00094a0 con 0x7f95ac0d2320 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95ab7fe700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c480 tx=0x7f959c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c005d70 con 0x7f95ac0d2320 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0070d0 con 0x7f95ac0d2320 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0d15e0 con 0x7f95ac0d2320 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c007440 con 0x7f95ac0d2320 2023-11-28T11:20:51.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.761+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0d1300 con 0x7f95ac0d2320 2023-11-28T11:20:51.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.762+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac15cd80 con 0x7f95ac0d2320 2023-11-28T11:20:51.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.763+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f959c019030 con 0x7f95ac0d2320 2023-11-28T11:20:51.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.764+0000 7f9593fff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 0x7f959806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.764+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f959c085060 con 0x7f95ac0d2320 2023-11-28T11:20:51.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.765+0000 7f95aaffd700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 0x7f959806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.766+0000 7f95aaffd700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 0x7f959806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f95a00090b0 tx=0x7f95a0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:51.769+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f959c04b5e0 con 0x7f95ac0d2320 2023-11-28T11:20:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.014+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95ac0745c0 con 0x7f95ac0d2320 2023-11-28T11:20:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.014+0000 7f9593fff700 1 -- 172.21.15.53:0/254459439 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f959c051880 con 0x7f95ac0d2320 2023-11-28T11:20:52.016 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:52.016 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:20:51.523382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.016+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 msgr2=0x7f959806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.016+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 0x7f959806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f95a00090b0 tx=0x7f95a0008030 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.016+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 msgr2=0x7f95ac150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.016+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c480 tx=0x7f959c00c840 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 shutdown_connections 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f95980682f0 0x7f959806a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac161340 0x7f95ac1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95ac0d2320 0x7f95ac150490 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 --2- 172.21.15.53:0/254459439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95ac15c9b0 0x7f95ac1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 >> 172.21.15.53:0/254459439 conn(0x7f95ac072600 msgr2=0x7f95ac0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 shutdown_connections 2023-11-28T11:20:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.017+0000 7f95b21ff700 1 -- 172.21.15.53:0/254459439 wait complete. 2023-11-28T11:20:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 32 2023-11-28T11:20:52.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:52.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.238+0000 7f9f155ff700 1 Processor -- start 2023-11-28T11:20:52.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.238+0000 7f9f155ff700 1 -- start start 2023-11-28T11:20:52.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f155ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f10161340 0x7f9f100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f155ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100d1950 con 0x7f9f100d2320 2023-11-28T11:20:52.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f155ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100b5520 con 0x7f9f1015c9b0 2023-11-28T11:20:52.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f155ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f100b5800 con 0x7f9f10161340 2023-11-28T11:20:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f0f7fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9f100d2320 0x7f9f100cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36282/0 (socket says 172.21.15.53:36282) 2023-11-28T11:20:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.239+0000 7f9f0f7fe700 1 -- 172.21.15.53:0/1815382752 learned_addr learned my addr 172.21.15.53:0/1815382752 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0e7fc700 1 --2- 172.21.15.53:0/1815382752 >> v2:172.21.15.157:3300/0 conn(0x7f9f10161340 0x7f9f100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093476955 0 0) 0x7f9f100d1950 con 0x7f9f100d2320 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef80035b0 con 0x7f9f100d2320 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504759412 0 0) 0x7f9ef80035b0 con 0x7f9f100d2320 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0e7fc700 1 -- 172.21.15.53:0/1815382752 >> v1:172.21.15.157:6789/0 conn(0x7f9f1015c9b0 legacy=0x7f9f100b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0e7fc700 1 -- 172.21.15.53:0/1815382752 >> v1:172.21.15.53:6789/0 conn(0x7f9f100d2320 legacy=0x7f9f100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.240+0000 7f9f0e7fc700 1 -- 172.21.15.53:0/1815382752 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100d1300 con 0x7f9f10161340 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f0e7fc700 1 --2- 172.21.15.53:0/1815382752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10161340 0x7f9f100d1de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009840 tx=0x7f9f04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=591daac270f645bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080019f0 con 0x7f9f10161340 2023-11-28T11:20:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f04004030 con 0x7f9f10161340 2023-11-28T11:20:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f0dffb700 1 -- 172.21.15.53:0/1815382752 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f040043c0 con 0x7f9f10161340 2023-11-28T11:20:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f155ff700 1 -- 172.21.15.53:0/1815382752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10161340 msgr2=0x7f9f100d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f155ff700 1 --2- 172.21.15.53:0/1815382752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10161340 0x7f9f100d1de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009840 tx=0x7f9f04009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f155ff700 1 -- 172.21.15.53:0/1815382752 shutdown_connections 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f155ff700 1 --2- 172.21.15.53:0/1815382752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f10161340 0x7f9f100d1de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.241+0000 7f9f155ff700 1 -- 172.21.15.53:0/1815382752 >> 172.21.15.53:0/1815382752 conn(0x7f9f10072600 msgr2=0x7f9f100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.242+0000 7f9f155ff700 1 -- 172.21.15.53:0/1815382752 shutdown_connections 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.242+0000 7f9f155ff700 1 -- 172.21.15.53:0/1815382752 wait complete. 2023-11-28T11:20:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.244+0000 7f9f155ff700 1 Processor -- start 2023-11-28T11:20:52.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.244+0000 7f9f155ff700 1 -- start start 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f100d2320 0x7f9f100c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f1015c9b0 0x7f9f101e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100c7bf0 con 0x7f9f10161340 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101e7070 con 0x7f9f100d2320 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.245+0000 7f9f155ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101e6460 con 0x7f9f1015c9b0 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38152/0 (socket says 172.21.15.53:38152) 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 -- 172.21.15.53:0/3075492279 learned_addr learned my addr 172.21.15.53:0/3075492279 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0effd700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f100d2320 0x7f9f100c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0e7fc700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f1015c9b0 0x7f9f101e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 -- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f1015c9b0 msgr2=0x7f9f101e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f1015c9b0 0x7f9f101e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 -- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f100d2320 msgr2=0x7f9f100c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f100d2320 0x7f9f100c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 -- 172.21.15.53:0/3075492279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040094a0 con 0x7f9f10161340 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.246+0000 7f9f0f7fe700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001560 tx=0x7f9f0800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.247+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800aaa0 con 0x7f9f10161340 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.247+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0800ac00 con 0x7f9f10161340 2023-11-28T11:20:52.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.247+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080070d0 con 0x7f9f10161340 2023-11-28T11:20:52.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.247+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1015cd80 con 0x7f9f10161340 2023-11-28T11:20:52.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.247+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f100d15e0 con 0x7f9f10161340 2023-11-28T11:20:52.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.248+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1015cf10 con 0x7f9f10161340 2023-11-28T11:20:52.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.249+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f08010070 con 0x7f9f10161340 2023-11-28T11:20:52.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.250+0000 7f9ef77fe700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.250+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f080856c0 con 0x7f9f10161340 2023-11-28T11:20:52.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.250+0000 7f9f0effd700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.251+0000 7f9f0effd700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 0x7f9efc06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007fd0 tx=0x7f9f00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.254+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f08051e30 con 0x7f9f10161340 2023-11-28T11:20:52.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.458+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f100745c0 con 0x7f9f10161340 2023-11-28T11:20:52.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.459+0000 7f9ef77fe700 1 -- 172.21.15.53:0/3075492279 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17921 (secure 0 0 0) 0x7f9f08055450 con 0x7f9f10161340 2023-11-28T11:20:52.462 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:52.462 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:20:49.511582+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.461+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 msgr2=0x7f9efc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.461+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 0x7f9efc06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007fd0 tx=0x7f9f00009040 comp rx=0 tx=0).stop 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 msgr2=0x7f9f10157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001560 tx=0x7f9f0800cf20 comp rx=0 tx=0).stop 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 shutdown_connections 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f10161340 0x7f9f10157580 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f100d2320 0x7f9f100c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 --2- 172.21.15.53:0/3075492279 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f1015c9b0 0x7f9f101e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 >> 172.21.15.53:0/3075492279 conn(0x7f9f10072600 msgr2=0x7f9f1015f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 shutdown_connections 2023-11-28T11:20:52.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.462+0000 7f9f155ff700 1 -- 172.21.15.53:0/3075492279 wait complete. 2023-11-28T11:20:52.483 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:20:52.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:20:52.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.683+0000 7f9df41ff700 1 Processor -- start 2023-11-28T11:20:52.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.683+0000 7f9df41ff700 1 -- start start 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dec15c9b0 0x7f9dec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec161340 0x7f9dec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9dec0d2320 0x7f9dec0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec0b5800 con 0x7f9dec0d2320 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec0d1950 con 0x7f9dec15c9b0 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.684+0000 7f9df41ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0b5520 con 0x7f9dec161340 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9dec0d2320 0x7f9dec0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9dec0d2320 0x7f9dec0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38168/0 (socket says 172.21.15.53:38168) 2023-11-28T11:20:52.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 -- 172.21.15.53:0/2972766358 learned_addr learned my addr 172.21.15.53:0/2972766358 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df179a700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec161340 0x7f9dec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df1f9b700 1 --2- 172.21.15.53:0/2972766358 >> v2:172.21.15.157:3300/0 conn(0x7f9dec15c9b0 0x7f9dec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 -- 172.21.15.53:0/2972766358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec161340 msgr2=0x7f9dec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec161340 0x7f9dec0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 -- 172.21.15.53:0/2972766358 >> v2:172.21.15.157:3300/0 conn(0x7f9dec15c9b0 msgr2=0x7f9dec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 --2- 172.21.15.53:0/2972766358 >> v2:172.21.15.157:3300/0 conn(0x7f9dec15c9b0 0x7f9dec0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.685+0000 7f9df279c700 1 -- 172.21.15.53:0/2972766358 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0d1300 con 0x7f9dec0d2320 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df279c700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec0cd7c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9dec067460 tx=0x7f9de400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2241261c3d55fe87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df0f99700 1 -- 172.21.15.53:0/2972766358 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400e070 con 0x7f9dec0d2320 2023-11-28T11:20:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df0f99700 1 -- 172.21.15.53:0/2972766358 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de400b910 con 0x7f9dec0d2320 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df0f99700 1 -- 172.21.15.53:0/2972766358 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400bcc0 con 0x7f9dec0d2320 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df41ff700 1 -- 172.21.15.53:0/2972766358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 msgr2=0x7f9dec0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.686+0000 7f9df41ff700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec0cd7c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9dec067460 tx=0x7f9de400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 -- 172.21.15.53:0/2972766358 shutdown_connections 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec0cd7c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 --2- 172.21.15.53:0/2972766358 >> v2:172.21.15.157:3300/0 conn(0x7f9dec15c9b0 0x7f9dec0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 --2- 172.21.15.53:0/2972766358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec161340 0x7f9dec0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 -- 172.21.15.53:0/2972766358 >> 172.21.15.53:0/2972766358 conn(0x7f9dec072600 msgr2=0x7f9dec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 -- 172.21.15.53:0/2972766358 shutdown_connections 2023-11-28T11:20:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.687+0000 7f9df41ff700 1 -- 172.21.15.53:0/2972766358 wait complete. 2023-11-28T11:20:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.688+0000 7f9df41ff700 1 Processor -- start 2023-11-28T11:20:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.689+0000 7f9df41ff700 1 -- start start 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec15c9b0 0x7f9dec151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dec161340 0x7f9dec15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1559b0 con 0x7f9dec0d2320 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec14dfb0 con 0x7f9dec161340 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df41ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec154da0 con 0x7f9dec15c9b0 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38176/0 (socket says 172.21.15.53:38176) 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 -- 172.21.15.53:0/4058647225 learned_addr learned my addr 172.21.15.53:0/4058647225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:52.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df179a700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec15c9b0 0x7f9dec151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df279c700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dec161340 0x7f9dec15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 -- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec15c9b0 msgr2=0x7f9dec151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec15c9b0 0x7f9dec151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 -- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dec161340 msgr2=0x7f9dec15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dec161340 0x7f9dec15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 -- 172.21.15.53:0/4058647225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009b30 con 0x7f9dec0d2320 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.690+0000 7f9df1f9b700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9de800e8a0 tx=0x7f9de800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.691+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8009590 con 0x7f9dec0d2320 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.691+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de800eea0 con 0x7f9dec0d2320 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.691+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800ca50 con 0x7f9dec0d2320 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.691+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec0b5130 con 0x7f9dec0d2320 2023-11-28T11:20:52.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.691+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec0d15e0 con 0x7f9dec0d2320 2023-11-28T11:20:52.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.692+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec0b52c0 con 0x7f9dec0d2320 2023-11-28T11:20:52.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.693+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9de8023070 con 0x7f9dec0d2320 2023-11-28T11:20:52.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.694+0000 7f9de27fc700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 0x7f9dd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.694+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9de8085eb0 con 0x7f9dec0d2320 2023-11-28T11:20:52.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.694+0000 7f9df179a700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 0x7f9dd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.695+0000 7f9df179a700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 0x7f9dd006a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0ce3a0 tx=0x7f9ddc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.698+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9de8052620 con 0x7f9dec0d2320 2023-11-28T11:20:52.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.899+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dec155af0 con 0x7f9dd00682f0 2023-11-28T11:20:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.909+0000 7f9de27fc700 1 -- 172.21.15.53:0/4058647225 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296859 (secure 0 0 0) 0x7f9ddc011720 con 0x7f9dd00682f0 2023-11-28T11:20:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 msgr2=0x7f9dd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 0x7f9dd006a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0ce3a0 tx=0x7f9ddc008040 comp rx=0 tx=0).stop 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 msgr2=0x7f9dec14da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9de800e8a0 tx=0x7f9de800ec60 comp rx=0 tx=0).stop 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 shutdown_connections 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9dd00682f0 0x7f9dd006a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dec161340 0x7f9dec15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dec0d2320 0x7f9dec14da70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 --2- 172.21.15.53:0/4058647225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dec15c9b0 0x7f9dec151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 >> 172.21.15.53:0/4058647225 conn(0x7f9dec072600 msgr2=0x7f9dec06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 shutdown_connections 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:52.914+0000 7f9df41ff700 1 -- 172.21.15.53:0/4058647225 wait complete. 2023-11-28T11:20:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:20:52.937 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-11-28T11:20:50.977583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":109568,"kb_used_data":6464,"kb_used_omap":0,"kb_used_meta":103040,"kb_avail":754865152,"statfs":{"total":773094113280,"available":772981915648,"internally_reserved":0,"allocated":6619136,"data_stored":3531729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":105512960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001113"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.110126+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:20:43.110126+0000","last_peered":"2023-11-28T11:20:43.110126+0000","last_clean":"2023-11-28T11:20:43.110126+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:20:43.110126+0000","last_undegraded":"2023-11-28T11:20:43.110126+0000","last_fullsized":"2023-11-28T11:20:43.110126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.109522+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:20:43.109522+0000","last_peered":"2023-11-28T11:20:43.109522+0000","last_clean":"2023-11-28T11:20:43.109522+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:20:43.109522+0000","last_undegraded":"2023-11-28T11:20:43.109522+0000","last_fullsized":"2023-11-28T11:20:43.109522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.110579+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:20:43.110579+0000","last_peered":"2023-11-28T11:20:43.110579+0000","last_clean":"2023-11-28T11:20:43.110579+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:20:43.110579+0000","last_undegraded":"2023-11-28T11:20:43.110579+0000","last_fullsized":"2023-11-28T11:20:43.110579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.109521+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:20:43.109521+0000","last_peered":"2023-11-28T11:20:43.109521+0000","last_clean":"2023-11-28T11:20:43.109521+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:20:43.109521+0000","last_undegraded":"2023-11-28T11:20:43.109521+0000","last_fullsized":"2023-11-28T11:20:43.109521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:42.537045+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:20:42.537045+0000","last_peered":"2023-11-28T11:20:42.537045+0000","last_clean":"2023-11-28T11:20:42.537045+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:20:42.537045+0000","last_undegraded":"2023-11-28T11:20:42.537045+0000","last_fullsized":"2023-11-28T11:20:42.537045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:42.537013+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:20:42.537013+0000","last_peered":"2023-11-28T11:20:42.537013+0000","last_clean":"2023-11-28T11:20:42.537013+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:20:42.537013+0000","last_undegraded":"2023-11-28T11:20:42.537013+0000","last_fullsized":"2023-11-28T11:20:42.537013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:42.537013+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:20:42.537013+0000","last_peered":"2023-11-28T11:20:42.537013+0000","last_clean":"2023-11-28T11:20:42.537013+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:20:42.537013+0000","last_undegraded":"2023-11-28T11:20:42.537013+0000","last_fullsized":"2023-11-28T11:20:42.537013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.110139+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:20:43.110139+0000","last_peered":"2023-11-28T11:20:43.110139+0000","last_clean":"2023-11-28T11:20:43.110139+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:20:43.110139+0000","last_undegraded":"2023-11-28T11:20:43.110139+0000","last_fullsized":"2023-11-28T11:20:43.110139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:47.995404+0000","last_change":"2023-11-28T11:20:47.995404+0000","last_active":"2023-11-28T11:20:47.995404+0000","last_peered":"2023-11-28T11:20:47.995404+0000","last_clean":"2023-11-28T11:20:47.995404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:47.995404+0000","last_undegraded":"2023-11-28T11:20:47.995404+0000","last_fullsized":"2023-11-28T11:20:47.995404+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:20:46.989015+0000","last_change":"2023-11-28T11:20:46.989015+0000","last_active":"2023-11-28T11:20:46.989015+0000","last_peered":"2023-11-28T11:20:46.989015+0000","last_clean":"2023-11-28T11:20:46.989015+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:20:46.989015+0000","last_undegraded":"2023-11-28T11:20:46.989015+0000","last_fullsized":"2023-11-28T11:20:46.989015+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:20:43.066200+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:20:43.066200+0000","last_peered":"2023-11-28T11:20:43.066200+0000","last_clean":"2023-11-28T11:20:43.066200+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:20:43.066200+0000","last_undegraded":"2023-11-28T11:20:43.066200+0000","last_fullsized":"2023-11-28T11:20:43.066200+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14564,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357276,"statfs":{"total":96636764160,"available":96621850624,"internally_reserved":0,"allocated":1273856,"data_stored":884490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78700000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88400000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":1.018,"15min":1.018},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.627}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.69299999999999995},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.92800000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96799999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97699999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81000000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13216,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358624,"statfs":{"total":96636764160,"available":96623230976,"internally_reserved":0,"allocated":679936,"data_stored":294122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.071,"5min":2.23,"15min":2.23},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78000000000000003,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95899999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89300000000000002,"5min":1.147,"15min":1.147},"last":0.67600000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.66800000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93500000000000005,"5min":1.224,"15min":1.224},"last":0.626},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.57899999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0029999999999999,"5min":1.627,"15min":1.627},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94199999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96499999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13340,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358500,"statfs":{"total":96636764160,"available":96623104000,"internally_reserved":0,"allocated":675840,"data_stored":293158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56100000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46400000000000002,"15min":0.46700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.376}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95999999999999996,"5min":1.181,"15min":1.181},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.57599999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86299999999999999,"5min":1.774,"15min":1.774},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.753},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97899999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.052,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90700000000000003,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72299999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14496,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357344,"statfs":{"total":96636764160,"available":96621920256,"internally_reserved":0,"allocated":1269760,"data_stored":883471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61199999999999999,"5min":1.052,"15min":1.052},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68200000000000005,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69199999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54500000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.056,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.163,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.022,"5min":1.458,"15min":1.458},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.98299999999999998,"5min":1.026,"15min":1.026},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.58899999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.93400000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.129,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1419999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80500000000000005}]}]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13344,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358496,"statfs":{"total":96636764160,"available":96623099904,"internally_reserved":0,"allocated":679936,"data_stored":294122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.60599999999999998,"15min":0.63500000000000001},"min":{"1min":0.28499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.58799999999999997,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":1.04,"15min":1.04},"last":0.41599999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.624,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.373},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.60099999999999998,"15min":0.625},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.67800000000000005,"15min":0.71999999999999997},"min":{"1min":0.318,"5min":0.317,"15min":0.317},"max":{"1min":0.54800000000000004,"5min":4.093,"15min":4.093},"last":0.44},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.78300000000000003,"15min":0.82499999999999996},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":3.956,"15min":3.956},"last":0.45300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.81399999999999995,"15min":0.83599999999999997},"min":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88300000000000001,"5min":2.028,"15min":2.028},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.76500000000000001,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.85599999999999998,"5min":1.147,"15min":1.147},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.94399999999999995,"15min":0.98899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.878,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.94899999999999995,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.93700000000000006,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.63}]},{"osd":6,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.88300000000000001,"15min":0.90500000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":1.016,"15min":1.0609999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96999999999999997,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0069999999999999,"5min":1.272,"15min":1.272},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.98299999999999998,"15min":1.0269999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.88100000000000001,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.58999999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13664,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358176,"statfs":{"total":96636764160,"available":96622772224,"internally_reserved":0,"allocated":679936,"data_stored":294122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.51500000000000001,"15min":0.55200000000000005},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.32100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.58899999999999997,"15min":0.63800000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":3.004,"15min":3.004},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63100000000000001,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66400000000000003,"5min":3.274,"15min":3.274},"last":0.46800000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48099999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.02,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94999999999999996,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.755},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.006,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.998,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81100000000000005,"15min":0.80700000000000005},"min":{"1min":0.67700000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82599999999999996,"15min":0.83399999999999996},"min":{"1min":0.65900000000000003,"5min":0.622,"15min":0.622},"max":{"1min":0.98299999999999998,"5min":1.099,"15min":1.099},"last":0.84299999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":13536,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358304,"statfs":{"total":96636764160,"available":96622903296,"internally_reserved":0,"allocated":679936,"data_stored":294122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.61399999999999999,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85599999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.53600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.82499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.46100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77600000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.77500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.47599999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.74199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.623,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.873,"5min":1.847,"15min":1.847},"last":0.84299999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94099999999999995,"5min":1.615,"15min":1.615},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":1.127,"15min":1.127},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86799999999999999,"5min":1.089,"15min":1.089},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78000000000000003,"15min":0.79300000000000004},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94099999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.94099999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84899999999999998,"5min":1.296,"15min":1.296},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89300000000000002,"5min":1.212,"15min":1.212},"last":0.80000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13408,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":12736,"kb_avail":94358432,"statfs":{"total":96636764160,"available":96623034368,"internally_reserved":0,"allocated":679936,"data_stored":294122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13041664},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.68899999999999995,"15min":0.66500000000000004},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72499999999999998,"15min":0.73599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.367,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.3199999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":2.661,"15min":2.661},"last":0.622},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.64000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.375,"5min":5.375,"15min":5.375},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.73599999999999999,"15min":0.72499999999999998},"min":{"1min":0.439,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.76200000000000001,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.333,"15min":1.333},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.1859999999999999,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:20:52.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:20:52.981 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-11-28T11:20:52.979+0000 7fa7363f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:20:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.175+0000 7f73ac9ff700 1 Processor -- start 2023-11-28T11:20:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.175+0000 7f73ac9ff700 1 -- start start 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73a815c9b0 0x7f73a80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a8161340 0x7f73a80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a80d2320 0x7f73a80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a80b5800 con 0x7f73a80d2320 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a80d1950 con 0x7f73a815c9b0 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73ac9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80b5520 con 0x7f73a8161340 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73a6d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a80d2320 0x7f73a80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73a6d9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a80d2320 0x7f73a80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38188/0 (socket says 172.21.15.53:38188) 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.176+0000 7f73a6d9d700 1 -- 172.21.15.53:0/2047992980 learned_addr learned my addr 172.21.15.53:0/2047992980 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a659c700 1 --2- 172.21.15.53:0/2047992980 >> v2:172.21.15.157:3300/0 conn(0x7f73a815c9b0 0x7f73a80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f739ffff700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a8161340 0x7f73a80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 -- 172.21.15.53:0/2047992980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a8161340 msgr2=0x7f73a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a8161340 0x7f73a80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 -- 172.21.15.53:0/2047992980 >> v2:172.21.15.157:3300/0 conn(0x7f73a815c9b0 msgr2=0x7f73a80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/2047992980 >> v2:172.21.15.157:3300/0 conn(0x7f73a815c9b0 0x7f73a80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 -- 172.21.15.53:0/2047992980 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80d1300 con 0x7f73a80d2320 2023-11-28T11:20:53.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a80d2320 0x7f73a80cd7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73a8067460 tx=0x7f73a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fe2a3241127ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a5d9b700 1 -- 172.21.15.53:0/2047992980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000e070 con 0x7f73a80d2320 2023-11-28T11:20:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a5d9b700 1 -- 172.21.15.53:0/2047992980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a000b9c0 con 0x7f73a80d2320 2023-11-28T11:20:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.177+0000 7f73a5d9b700 1 -- 172.21.15.53:0/2047992980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000bd70 con 0x7f73a80d2320 2023-11-28T11:20:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 -- 172.21.15.53:0/2047992980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a80d2320 msgr2=0x7f73a80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a80d2320 0x7f73a80cd7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73a8067460 tx=0x7f73a000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 -- 172.21.15.53:0/2047992980 shutdown_connections 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a80d2320 0x7f73a80cd7c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/2047992980 >> v2:172.21.15.157:3300/0 conn(0x7f73a815c9b0 0x7f73a80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/2047992980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a8161340 0x7f73a80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 -- 172.21.15.53:0/2047992980 >> 172.21.15.53:0/2047992980 conn(0x7f73a8072600 msgr2=0x7f73a80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 -- 172.21.15.53:0/2047992980 shutdown_connections 2023-11-28T11:20:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.178+0000 7f73ac9ff700 1 -- 172.21.15.53:0/2047992980 wait complete. 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.180+0000 7f73ac9ff700 1 Processor -- start 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.180+0000 7f73ac9ff700 1 -- start start 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a80d2320 0x7f73a815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a815c9b0 0x7f73a80c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80c6a00 con 0x7f73a8161340 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81f2680 con 0x7f73a815c9b0 2023-11-28T11:20:53.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73ac9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81559a0 con 0x7f73a80d2320 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a659c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a80d2320 0x7f73a815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38200/0 (socket says 172.21.15.53:38200) 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 -- 172.21.15.53:0/998032874 learned_addr learned my addr 172.21.15.53:0/998032874 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f739ffff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a815c9b0 0x7f73a80c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 -- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a80d2320 msgr2=0x7f73a815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a80d2320 0x7f73a815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 -- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a815c9b0 msgr2=0x7f73a80c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.181+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a815c9b0 0x7f73a80c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f73a6d9d700 1 -- 172.21.15.53:0/998032874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0009b30 con 0x7f73a8161340 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f73a6d9d700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73a80675d0 tx=0x7f73a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00099b0 con 0x7f73a8161340 2023-11-28T11:20:53.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0004250 con 0x7f73a8161340 2023-11-28T11:20:53.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004600 con 0x7f73a8161340 2023-11-28T11:20:53.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a80b5130 con 0x7f73a8161340 2023-11-28T11:20:53.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.182+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a80d15e0 con 0x7f73a8161340 2023-11-28T11:20:53.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.183+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a80b52c0 con 0x7f73a8161340 2023-11-28T11:20:53.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.184+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73a002b070 con 0x7f73a8161340 2023-11-28T11:20:53.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.185+0000 7f739effd700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 0x7f738806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.185+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73a0086f00 con 0x7f73a8161340 2023-11-28T11:20:53.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.185+0000 7f73a659c700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 0x7f738806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.186+0000 7f73a659c700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 0x7f738806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7398006b20 tx=0x7f7398008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.189+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73a0053760 con 0x7f73a8161340 2023-11-28T11:20:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.388+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73a80745c0 con 0x7f73880682f0 2023-11-28T11:20:53.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.397+0000 7f739effd700 1 -- 172.21.15.53:0/998032874 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311799 (secure 0 0 0) 0x7f73a80745c0 con 0x7f73880682f0 2023-11-28T11:20:53.403 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 msgr2=0x7f738806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 0x7f738806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7398006b20 tx=0x7f7398008040 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 msgr2=0x7f73a80c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73a80675d0 tx=0x7f73a0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 shutdown_connections 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f73880682f0 0x7f738806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73a80d2320 0x7f73a815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a8161340 0x7f73a80c64c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 --2- 172.21.15.53:0/998032874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73a815c9b0 0x7f73a80c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 >> 172.21.15.53:0/998032874 conn(0x7f73a8072600 msgr2=0x7f73a815d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 shutdown_connections 2023-11-28T11:20:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.402+0000 7f73ac9ff700 1 -- 172.21.15.53:0/998032874 wait complete. 2023-11-28T11:20:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:20:53.422 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2023-11-28T11:20:52.977999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":111296,"kb_used_data":6912,"kb_used_omap":0,"kb_used_meta":104320,"kb_avail":754863424,"statfs":{"total":773094113280,"available":772980146176,"internally_reserved":0,"allocated":7077888,"data_stored":3748957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":106823680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.981203"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765132+0000","last_change":"2023-11-28T11:20:48.529416+0000","last_active":"2023-11-28T11:20:49.765132+0000","last_peered":"2023-11-28T11:20:49.765132+0000","last_clean":"2023-11-28T11:20:49.765132+0000","last_became_active":"2023-11-28T11:20:48.529058+0000","last_became_peered":"2023-11-28T11:20:48.529058+0000","last_unstale":"2023-11-28T11:20:49.765132+0000","last_undegraded":"2023-11-28T11:20:49.765132+0000","last_fullsized":"2023-11-28T11:20:49.765132+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.835275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764899+0000","last_change":"2023-11-28T11:20:48.533368+0000","last_active":"2023-11-28T11:20:49.764899+0000","last_peered":"2023-11-28T11:20:49.764899+0000","last_clean":"2023-11-28T11:20:49.764899+0000","last_became_active":"2023-11-28T11:20:48.533188+0000","last_became_peered":"2023-11-28T11:20:48.533188+0000","last_unstale":"2023-11-28T11:20:49.764899+0000","last_undegraded":"2023-11-28T11:20:49.764899+0000","last_fullsized":"2023-11-28T11:20:49.764899+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:23.089940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765647+0000","last_change":"2023-11-28T11:20:48.548684+0000","last_active":"2023-11-28T11:20:49.765647+0000","last_peered":"2023-11-28T11:20:49.765647+0000","last_clean":"2023-11-28T11:20:49.765647+0000","last_became_active":"2023-11-28T11:20:48.546917+0000","last_became_peered":"2023-11-28T11:20:48.546917+0000","last_unstale":"2023-11-28T11:20:49.765647+0000","last_undegraded":"2023-11-28T11:20:49.765647+0000","last_fullsized":"2023-11-28T11:20:49.765647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:17.017413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520422+0000","last_change":"2023-11-28T11:20:48.521832+0000","last_active":"2023-11-28T11:20:49.520422+0000","last_peered":"2023-11-28T11:20:49.520422+0000","last_clean":"2023-11-28T11:20:49.520422+0000","last_became_active":"2023-11-28T11:20:48.521550+0000","last_became_peered":"2023-11-28T11:20:48.521550+0000","last_unstale":"2023-11-28T11:20:49.520422+0000","last_undegraded":"2023-11-28T11:20:49.520422+0000","last_fullsized":"2023-11-28T11:20:49.520422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:34.965120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765975+0000","last_change":"2023-11-28T11:20:48.538435+0000","last_active":"2023-11-28T11:20:49.765975+0000","last_peered":"2023-11-28T11:20:49.765975+0000","last_clean":"2023-11-28T11:20:49.765975+0000","last_became_active":"2023-11-28T11:20:48.538156+0000","last_became_peered":"2023-11-28T11:20:48.538156+0000","last_unstale":"2023-11-28T11:20:49.765975+0000","last_undegraded":"2023-11-28T11:20:49.765975+0000","last_fullsized":"2023-11-28T11:20:49.765975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:49.680614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738387+0000","last_change":"2023-11-28T11:20:48.548659+0000","last_active":"2023-11-28T11:20:49.738387+0000","last_peered":"2023-11-28T11:20:49.738387+0000","last_clean":"2023-11-28T11:20:49.738387+0000","last_became_active":"2023-11-28T11:20:48.548384+0000","last_became_peered":"2023-11-28T11:20:48.548384+0000","last_unstale":"2023-11-28T11:20:49.738387+0000","last_undegraded":"2023-11-28T11:20:49.738387+0000","last_fullsized":"2023-11-28T11:20:49.738387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:03:19.288700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764544+0000","last_change":"2023-11-28T11:20:48.551930+0000","last_active":"2023-11-28T11:20:49.764544+0000","last_peered":"2023-11-28T11:20:49.764544+0000","last_clean":"2023-11-28T11:20:49.764544+0000","last_became_active":"2023-11-28T11:20:48.551412+0000","last_became_peered":"2023-11-28T11:20:48.551412+0000","last_unstale":"2023-11-28T11:20:49.764544+0000","last_undegraded":"2023-11-28T11:20:49.764544+0000","last_fullsized":"2023-11-28T11:20:49.764544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:13.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520966+0000","last_change":"2023-11-28T11:20:48.540869+0000","last_active":"2023-11-28T11:20:49.520966+0000","last_peered":"2023-11-28T11:20:49.520966+0000","last_clean":"2023-11-28T11:20:49.520966+0000","last_became_active":"2023-11-28T11:20:48.540417+0000","last_became_peered":"2023-11-28T11:20:48.540417+0000","last_unstale":"2023-11-28T11:20:49.520966+0000","last_undegraded":"2023-11-28T11:20:49.520966+0000","last_fullsized":"2023-11-28T11:20:49.520966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:27.041551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766108+0000","last_change":"2023-11-28T11:20:48.547699+0000","last_active":"2023-11-28T11:20:49.766108+0000","last_peered":"2023-11-28T11:20:49.766108+0000","last_clean":"2023-11-28T11:20:49.766108+0000","last_became_active":"2023-11-28T11:20:48.547363+0000","last_became_peered":"2023-11-28T11:20:48.547363+0000","last_unstale":"2023-11-28T11:20:49.766108+0000","last_undegraded":"2023-11-28T11:20:49.766108+0000","last_fullsized":"2023-11-28T11:20:49.766108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:00.065606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766249+0000","last_change":"2023-11-28T11:20:48.542457+0000","last_active":"2023-11-28T11:20:49.766249+0000","last_peered":"2023-11-28T11:20:49.766249+0000","last_clean":"2023-11-28T11:20:49.766249+0000","last_became_active":"2023-11-28T11:20:48.542065+0000","last_became_peered":"2023-11-28T11:20:48.542065+0000","last_unstale":"2023-11-28T11:20:49.766249+0000","last_undegraded":"2023-11-28T11:20:49.766249+0000","last_fullsized":"2023-11-28T11:20:49.766249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:16.626998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520211+0000","last_change":"2023-11-28T11:20:48.535853+0000","last_active":"2023-11-28T11:20:49.520211+0000","last_peered":"2023-11-28T11:20:49.520211+0000","last_clean":"2023-11-28T11:20:49.520211+0000","last_became_active":"2023-11-28T11:20:48.535363+0000","last_became_peered":"2023-11-28T11:20:48.535363+0000","last_unstale":"2023-11-28T11:20:49.520211+0000","last_undegraded":"2023-11-28T11:20:49.520211+0000","last_fullsized":"2023-11-28T11:20:49.520211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.835275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520258+0000","last_change":"2023-11-28T11:20:48.547742+0000","last_active":"2023-11-28T11:20:49.520258+0000","last_peered":"2023-11-28T11:20:49.520258+0000","last_clean":"2023-11-28T11:20:49.520258+0000","last_became_active":"2023-11-28T11:20:48.547451+0000","last_became_peered":"2023-11-28T11:20:48.547451+0000","last_unstale":"2023-11-28T11:20:49.520258+0000","last_undegraded":"2023-11-28T11:20:49.520258+0000","last_fullsized":"2023-11-28T11:20:49.520258+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:52:13.404941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520005+0000","last_change":"2023-11-28T11:20:48.529415+0000","last_active":"2023-11-28T11:20:49.520005+0000","last_peered":"2023-11-28T11:20:49.520005+0000","last_clean":"2023-11-28T11:20:49.520005+0000","last_became_active":"2023-11-28T11:20:48.529136+0000","last_became_peered":"2023-11-28T11:20:48.529136+0000","last_unstale":"2023-11-28T11:20:49.520005+0000","last_undegraded":"2023-11-28T11:20:49.520005+0000","last_fullsized":"2023-11-28T11:20:49.520005+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:24.199496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765878+0000","last_change":"2023-11-28T11:20:48.548073+0000","last_active":"2023-11-28T11:20:49.765878+0000","last_peered":"2023-11-28T11:20:49.765878+0000","last_clean":"2023-11-28T11:20:49.765878+0000","last_became_active":"2023-11-28T11:20:48.547799+0000","last_became_peered":"2023-11-28T11:20:48.547799+0000","last_unstale":"2023-11-28T11:20:49.765878+0000","last_undegraded":"2023-11-28T11:20:49.765878+0000","last_fullsized":"2023-11-28T11:20:49.765878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:36.801615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765382+0000","last_change":"2023-11-28T11:20:48.550799+0000","last_active":"2023-11-28T11:20:49.765382+0000","last_peered":"2023-11-28T11:20:49.765382+0000","last_clean":"2023-11-28T11:20:49.765382+0000","last_became_active":"2023-11-28T11:20:48.550445+0000","last_became_peered":"2023-11-28T11:20:48.550445+0000","last_unstale":"2023-11-28T11:20:49.765382+0000","last_undegraded":"2023-11-28T11:20:49.765382+0000","last_fullsized":"2023-11-28T11:20:49.765382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:59.896316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764405+0000","last_change":"2023-11-28T11:20:48.533304+0000","last_active":"2023-11-28T11:20:49.764405+0000","last_peered":"2023-11-28T11:20:49.764405+0000","last_clean":"2023-11-28T11:20:49.764405+0000","last_became_active":"2023-11-28T11:20:48.533044+0000","last_became_peered":"2023-11-28T11:20:48.533044+0000","last_unstale":"2023-11-28T11:20:49.764405+0000","last_undegraded":"2023-11-28T11:20:49.764405+0000","last_fullsized":"2023-11-28T11:20:49.764405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:50.322117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765282+0000","last_change":"2023-11-28T11:20:48.526447+0000","last_active":"2023-11-28T11:20:49.765282+0000","last_peered":"2023-11-28T11:20:49.765282+0000","last_clean":"2023-11-28T11:20:49.765282+0000","last_became_active":"2023-11-28T11:20:48.526146+0000","last_became_peered":"2023-11-28T11:20:48.526146+0000","last_unstale":"2023-11-28T11:20:49.765282+0000","last_undegraded":"2023-11-28T11:20:49.765282+0000","last_fullsized":"2023-11-28T11:20:49.765282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:11.399024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520608+0000","last_change":"2023-11-28T11:20:48.529276+0000","last_active":"2023-11-28T11:20:49.520608+0000","last_peered":"2023-11-28T11:20:49.520608+0000","last_clean":"2023-11-28T11:20:49.520608+0000","last_became_active":"2023-11-28T11:20:48.528962+0000","last_became_peered":"2023-11-28T11:20:48.528962+0000","last_unstale":"2023-11-28T11:20:49.520608+0000","last_undegraded":"2023-11-28T11:20:49.520608+0000","last_fullsized":"2023-11-28T11:20:49.520608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:43.633668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765950+0000","last_change":"2023-11-28T11:20:48.535682+0000","last_active":"2023-11-28T11:20:49.765950+0000","last_peered":"2023-11-28T11:20:49.765950+0000","last_clean":"2023-11-28T11:20:49.765950+0000","last_became_active":"2023-11-28T11:20:48.535375+0000","last_became_peered":"2023-11-28T11:20:48.535375+0000","last_unstale":"2023-11-28T11:20:49.765950+0000","last_undegraded":"2023-11-28T11:20:49.765950+0000","last_fullsized":"2023-11-28T11:20:49.765950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:25:38.468180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518900+0000","last_change":"2023-11-28T11:20:48.526039+0000","last_active":"2023-11-28T11:20:49.518900+0000","last_peered":"2023-11-28T11:20:49.518900+0000","last_clean":"2023-11-28T11:20:49.518900+0000","last_became_active":"2023-11-28T11:20:48.525754+0000","last_became_peered":"2023-11-28T11:20:48.525754+0000","last_unstale":"2023-11-28T11:20:49.518900+0000","last_undegraded":"2023-11-28T11:20:49.518900+0000","last_fullsized":"2023-11-28T11:20:49.518900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:49.123097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519880+0000","last_change":"2023-11-28T11:20:48.532324+0000","last_active":"2023-11-28T11:20:49.519880+0000","last_peered":"2023-11-28T11:20:49.519880+0000","last_clean":"2023-11-28T11:20:49.519880+0000","last_became_active":"2023-11-28T11:20:48.532007+0000","last_became_peered":"2023-11-28T11:20:48.532007+0000","last_unstale":"2023-11-28T11:20:49.519880+0000","last_undegraded":"2023-11-28T11:20:49.519880+0000","last_fullsized":"2023-11-28T11:20:49.519880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:29.629639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518851+0000","last_change":"2023-11-28T11:20:48.543926+0000","last_active":"2023-11-28T11:20:49.518851+0000","last_peered":"2023-11-28T11:20:49.518851+0000","last_clean":"2023-11-28T11:20:49.518851+0000","last_became_active":"2023-11-28T11:20:48.543514+0000","last_became_peered":"2023-11-28T11:20:48.543514+0000","last_unstale":"2023-11-28T11:20:49.518851+0000","last_undegraded":"2023-11-28T11:20:49.518851+0000","last_fullsized":"2023-11-28T11:20:49.518851+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:30.157141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518616+0000","last_change":"2023-11-28T11:20:48.548839+0000","last_active":"2023-11-28T11:20:49.518616+0000","last_peered":"2023-11-28T11:20:49.518616+0000","last_clean":"2023-11-28T11:20:49.518616+0000","last_became_active":"2023-11-28T11:20:48.548504+0000","last_became_peered":"2023-11-28T11:20:48.548504+0000","last_unstale":"2023-11-28T11:20:49.518616+0000","last_undegraded":"2023-11-28T11:20:49.518616+0000","last_fullsized":"2023-11-28T11:20:49.518616+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:00.296286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765499+0000","last_change":"2023-11-28T11:20:48.545882+0000","last_active":"2023-11-28T11:20:49.765499+0000","last_peered":"2023-11-28T11:20:49.765499+0000","last_clean":"2023-11-28T11:20:49.765499+0000","last_became_active":"2023-11-28T11:20:48.542232+0000","last_became_peered":"2023-11-28T11:20:48.542232+0000","last_unstale":"2023-11-28T11:20:49.765499+0000","last_undegraded":"2023-11-28T11:20:49.765499+0000","last_fullsized":"2023-11-28T11:20:49.765499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:58.059136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520812+0000","last_change":"2023-11-28T11:20:48.538683+0000","last_active":"2023-11-28T11:20:49.520812+0000","last_peered":"2023-11-28T11:20:49.520812+0000","last_clean":"2023-11-28T11:20:49.520812+0000","last_became_active":"2023-11-28T11:20:48.538409+0000","last_became_peered":"2023-11-28T11:20:48.538409+0000","last_unstale":"2023-11-28T11:20:49.520812+0000","last_undegraded":"2023-11-28T11:20:49.520812+0000","last_fullsized":"2023-11-28T11:20:49.520812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:43.010054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520685+0000","last_change":"2023-11-28T11:20:48.533484+0000","last_active":"2023-11-28T11:20:49.520685+0000","last_peered":"2023-11-28T11:20:49.520685+0000","last_clean":"2023-11-28T11:20:49.520685+0000","last_became_active":"2023-11-28T11:20:48.533194+0000","last_became_peered":"2023-11-28T11:20:48.533194+0000","last_unstale":"2023-11-28T11:20:49.520685+0000","last_undegraded":"2023-11-28T11:20:49.520685+0000","last_fullsized":"2023-11-28T11:20:49.520685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:49.623350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520311+0000","last_change":"2023-11-28T11:20:48.536501+0000","last_active":"2023-11-28T11:20:49.520311+0000","last_peered":"2023-11-28T11:20:49.520311+0000","last_clean":"2023-11-28T11:20:49.520311+0000","last_became_active":"2023-11-28T11:20:48.536179+0000","last_became_peered":"2023-11-28T11:20:48.536179+0000","last_unstale":"2023-11-28T11:20:49.520311+0000","last_undegraded":"2023-11-28T11:20:49.520311+0000","last_fullsized":"2023-11-28T11:20:49.520311+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:17.109714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765082+0000","last_change":"2023-11-28T11:20:48.536478+0000","last_active":"2023-11-28T11:20:49.765082+0000","last_peered":"2023-11-28T11:20:49.765082+0000","last_clean":"2023-11-28T11:20:49.765082+0000","last_became_active":"2023-11-28T11:20:48.535329+0000","last_became_peered":"2023-11-28T11:20:48.535329+0000","last_unstale":"2023-11-28T11:20:49.765082+0000","last_undegraded":"2023-11-28T11:20:49.765082+0000","last_fullsized":"2023-11-28T11:20:49.765082+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:54.913676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765775+0000","last_change":"2023-11-28T11:20:48.549787+0000","last_active":"2023-11-28T11:20:49.765775+0000","last_peered":"2023-11-28T11:20:49.765775+0000","last_clean":"2023-11-28T11:20:49.765775+0000","last_became_active":"2023-11-28T11:20:48.549508+0000","last_became_peered":"2023-11-28T11:20:48.549508+0000","last_unstale":"2023-11-28T11:20:49.765775+0000","last_undegraded":"2023-11-28T11:20:49.765775+0000","last_fullsized":"2023-11-28T11:20:49.765775+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:36.952285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519759+0000","last_change":"2023-11-28T11:20:48.549196+0000","last_active":"2023-11-28T11:20:49.519759+0000","last_peered":"2023-11-28T11:20:49.519759+0000","last_clean":"2023-11-28T11:20:49.519759+0000","last_became_active":"2023-11-28T11:20:48.548887+0000","last_became_peered":"2023-11-28T11:20:48.548887+0000","last_unstale":"2023-11-28T11:20:49.519759+0000","last_undegraded":"2023-11-28T11:20:49.519759+0000","last_fullsized":"2023-11-28T11:20:49.519759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:45:54.593454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520734+0000","last_change":"2023-11-28T11:20:48.530119+0000","last_active":"2023-11-28T11:20:49.520734+0000","last_peered":"2023-11-28T11:20:49.520734+0000","last_clean":"2023-11-28T11:20:49.520734+0000","last_became_active":"2023-11-28T11:20:48.529829+0000","last_became_peered":"2023-11-28T11:20:48.529829+0000","last_unstale":"2023-11-28T11:20:49.520734+0000","last_undegraded":"2023-11-28T11:20:49.520734+0000","last_fullsized":"2023-11-28T11:20:49.520734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:20.148723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738189+0000","last_change":"2023-11-28T11:20:48.533670+0000","last_active":"2023-11-28T11:20:49.738189+0000","last_peered":"2023-11-28T11:20:49.738189+0000","last_clean":"2023-11-28T11:20:49.738189+0000","last_became_active":"2023-11-28T11:20:48.532711+0000","last_became_peered":"2023-11-28T11:20:48.532711+0000","last_unstale":"2023-11-28T11:20:49.738189+0000","last_undegraded":"2023-11-28T11:20:49.738189+0000","last_fullsized":"2023-11-28T11:20:49.738189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:16:57.593732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766068+0000","last_change":"2023-11-28T11:20:48.539982+0000","last_active":"2023-11-28T11:20:49.766068+0000","last_peered":"2023-11-28T11:20:49.766068+0000","last_clean":"2023-11-28T11:20:49.766068+0000","last_became_active":"2023-11-28T11:20:48.539343+0000","last_became_peered":"2023-11-28T11:20:48.539343+0000","last_unstale":"2023-11-28T11:20:49.766068+0000","last_undegraded":"2023-11-28T11:20:49.766068+0000","last_fullsized":"2023-11-28T11:20:49.766068+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:41.853635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765372+0000","last_change":"2023-11-28T11:20:48.029355+0000","last_active":"2023-11-28T11:20:49.765372+0000","last_peered":"2023-11-28T11:20:49.765372+0000","last_clean":"2023-11-28T11:20:49.765372+0000","last_became_active":"2023-11-28T11:20:48.029035+0000","last_became_peered":"2023-11-28T11:20:48.029035+0000","last_unstale":"2023-11-28T11:20:49.765372+0000","last_undegraded":"2023-11-28T11:20:49.765372+0000","last_fullsized":"2023-11-28T11:20:49.765372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:23.875674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520421+0000","last_change":"2023-11-28T11:20:48.535647+0000","last_active":"2023-11-28T11:20:49.520421+0000","last_peered":"2023-11-28T11:20:49.520421+0000","last_clean":"2023-11-28T11:20:49.520421+0000","last_became_active":"2023-11-28T11:20:48.534990+0000","last_became_peered":"2023-11-28T11:20:48.534990+0000","last_unstale":"2023-11-28T11:20:49.520421+0000","last_undegraded":"2023-11-28T11:20:49.520421+0000","last_fullsized":"2023-11-28T11:20:49.520421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:11.399024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738774+0000","last_change":"2023-11-28T11:20:48.027775+0000","last_active":"2023-11-28T11:20:49.738774+0000","last_peered":"2023-11-28T11:20:49.738774+0000","last_clean":"2023-11-28T11:20:49.738774+0000","last_became_active":"2023-11-28T11:20:48.027446+0000","last_became_peered":"2023-11-28T11:20:48.027446+0000","last_unstale":"2023-11-28T11:20:49.738774+0000","last_undegraded":"2023-11-28T11:20:49.738774+0000","last_fullsized":"2023-11-28T11:20:49.738774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:54.258477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738735+0000","last_change":"2023-11-28T11:20:48.534808+0000","last_active":"2023-11-28T11:20:49.738735+0000","last_peered":"2023-11-28T11:20:49.738735+0000","last_clean":"2023-11-28T11:20:49.738735+0000","last_became_active":"2023-11-28T11:20:48.534499+0000","last_became_peered":"2023-11-28T11:20:48.534499+0000","last_unstale":"2023-11-28T11:20:49.738735+0000","last_undegraded":"2023-11-28T11:20:49.738735+0000","last_fullsized":"2023-11-28T11:20:49.738735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:44.707386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738577+0000","last_change":"2023-11-28T11:20:48.024731+0000","last_active":"2023-11-28T11:20:49.738577+0000","last_peered":"2023-11-28T11:20:49.738577+0000","last_clean":"2023-11-28T11:20:49.738577+0000","last_became_active":"2023-11-28T11:20:48.024435+0000","last_became_peered":"2023-11-28T11:20:48.024435+0000","last_unstale":"2023-11-28T11:20:49.738577+0000","last_undegraded":"2023-11-28T11:20:49.738577+0000","last_fullsized":"2023-11-28T11:20:49.738577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:35.945895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765945+0000","last_change":"2023-11-28T11:20:48.545618+0000","last_active":"2023-11-28T11:20:49.765945+0000","last_peered":"2023-11-28T11:20:49.765945+0000","last_clean":"2023-11-28T11:20:49.765945+0000","last_became_active":"2023-11-28T11:20:48.545244+0000","last_became_peered":"2023-11-28T11:20:48.545244+0000","last_unstale":"2023-11-28T11:20:49.765945+0000","last_undegraded":"2023-11-28T11:20:49.765945+0000","last_fullsized":"2023-11-28T11:20:49.765945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:13.347401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738528+0000","last_change":"2023-11-28T11:20:48.014287+0000","last_active":"2023-11-28T11:20:49.738528+0000","last_peered":"2023-11-28T11:20:49.738528+0000","last_clean":"2023-11-28T11:20:49.738528+0000","last_became_active":"2023-11-28T11:20:48.013150+0000","last_became_peered":"2023-11-28T11:20:48.013150+0000","last_unstale":"2023-11-28T11:20:49.738528+0000","last_undegraded":"2023-11-28T11:20:49.738528+0000","last_fullsized":"2023-11-28T11:20:49.738528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:59.059216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764522+0000","last_change":"2023-11-28T11:20:48.531904+0000","last_active":"2023-11-28T11:20:49.764522+0000","last_peered":"2023-11-28T11:20:49.764522+0000","last_clean":"2023-11-28T11:20:49.764522+0000","last_became_active":"2023-11-28T11:20:48.531601+0000","last_became_peered":"2023-11-28T11:20:48.531601+0000","last_unstale":"2023-11-28T11:20:49.764522+0000","last_undegraded":"2023-11-28T11:20:49.764522+0000","last_fullsized":"2023-11-28T11:20:49.764522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:33.371357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518895+0000","last_change":"2023-11-28T11:20:48.012716+0000","last_active":"2023-11-28T11:20:49.518895+0000","last_peered":"2023-11-28T11:20:49.518895+0000","last_clean":"2023-11-28T11:20:49.518895+0000","last_became_active":"2023-11-28T11:20:48.012433+0000","last_became_peered":"2023-11-28T11:20:48.012433+0000","last_unstale":"2023-11-28T11:20:49.518895+0000","last_undegraded":"2023-11-28T11:20:49.518895+0000","last_fullsized":"2023-11-28T11:20:49.518895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:16.011023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764388+0000","last_change":"2023-11-28T11:20:48.530422+0000","last_active":"2023-11-28T11:20:49.764388+0000","last_peered":"2023-11-28T11:20:49.764388+0000","last_clean":"2023-11-28T11:20:49.764388+0000","last_became_active":"2023-11-28T11:20:48.530166+0000","last_became_peered":"2023-11-28T11:20:48.530166+0000","last_unstale":"2023-11-28T11:20:49.764388+0000","last_undegraded":"2023-11-28T11:20:49.764388+0000","last_fullsized":"2023-11-28T11:20:49.764388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:43.633668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518717+0000","last_change":"2023-11-28T11:20:48.017462+0000","last_active":"2023-11-28T11:20:49.518717+0000","last_peered":"2023-11-28T11:20:49.518717+0000","last_clean":"2023-11-28T11:20:49.518717+0000","last_became_active":"2023-11-28T11:20:48.017098+0000","last_became_peered":"2023-11-28T11:20:48.017098+0000","last_unstale":"2023-11-28T11:20:49.518717+0000","last_undegraded":"2023-11-28T11:20:49.518717+0000","last_fullsized":"2023-11-28T11:20:49.518717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:57.052746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765604+0000","last_change":"2023-11-28T11:20:48.538550+0000","last_active":"2023-11-28T11:20:49.765604+0000","last_peered":"2023-11-28T11:20:49.765604+0000","last_clean":"2023-11-28T11:20:49.765604+0000","last_became_active":"2023-11-28T11:20:48.537339+0000","last_became_peered":"2023-11-28T11:20:48.537339+0000","last_unstale":"2023-11-28T11:20:49.765604+0000","last_undegraded":"2023-11-28T11:20:49.765604+0000","last_fullsized":"2023-11-28T11:20:49.765604+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:37:50.042024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518585+0000","last_change":"2023-11-28T11:20:48.012880+0000","last_active":"2023-11-28T11:20:49.518585+0000","last_peered":"2023-11-28T11:20:49.518585+0000","last_clean":"2023-11-28T11:20:49.518585+0000","last_became_active":"2023-11-28T11:20:48.012648+0000","last_became_peered":"2023-11-28T11:20:48.012648+0000","last_unstale":"2023-11-28T11:20:49.518585+0000","last_undegraded":"2023-11-28T11:20:49.518585+0000","last_fullsized":"2023-11-28T11:20:49.518585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:19.142333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518710+0000","last_change":"2023-11-28T11:20:48.525544+0000","last_active":"2023-11-28T11:20:49.518710+0000","last_peered":"2023-11-28T11:20:49.518710+0000","last_clean":"2023-11-28T11:20:49.518710+0000","last_became_active":"2023-11-28T11:20:48.525306+0000","last_became_peered":"2023-11-28T11:20:48.525306+0000","last_unstale":"2023-11-28T11:20:49.518710+0000","last_undegraded":"2023-11-28T11:20:49.518710+0000","last_fullsized":"2023-11-28T11:20:49.518710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:36.952285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765528+0000","last_change":"2023-11-28T11:20:48.021278+0000","last_active":"2023-11-28T11:20:49.765528+0000","last_peered":"2023-11-28T11:20:49.765528+0000","last_clean":"2023-11-28T11:20:49.765528+0000","last_became_active":"2023-11-28T11:20:48.019655+0000","last_became_peered":"2023-11-28T11:20:48.019655+0000","last_unstale":"2023-11-28T11:20:49.765528+0000","last_undegraded":"2023-11-28T11:20:49.765528+0000","last_fullsized":"2023-11-28T11:20:49.765528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:38.615376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764971+0000","last_change":"2023-11-28T11:20:48.537976+0000","last_active":"2023-11-28T11:20:49.764971+0000","last_peered":"2023-11-28T11:20:49.764971+0000","last_clean":"2023-11-28T11:20:49.764971+0000","last_became_active":"2023-11-28T11:20:48.537638+0000","last_became_peered":"2023-11-28T11:20:48.537638+0000","last_unstale":"2023-11-28T11:20:49.764971+0000","last_undegraded":"2023-11-28T11:20:49.764971+0000","last_fullsized":"2023-11-28T11:20:49.764971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:34.965120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766036+0000","last_change":"2023-11-28T11:20:48.030034+0000","last_active":"2023-11-28T11:20:49.766036+0000","last_peered":"2023-11-28T11:20:49.766036+0000","last_clean":"2023-11-28T11:20:49.766036+0000","last_became_active":"2023-11-28T11:20:48.029002+0000","last_became_peered":"2023-11-28T11:20:48.029002+0000","last_unstale":"2023-11-28T11:20:49.766036+0000","last_undegraded":"2023-11-28T11:20:49.766036+0000","last_fullsized":"2023-11-28T11:20:49.766036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:13.549378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519030+0000","last_change":"2023-11-28T11:20:48.524805+0000","last_active":"2023-11-28T11:20:49.519030+0000","last_peered":"2023-11-28T11:20:49.519030+0000","last_clean":"2023-11-28T11:20:49.519030+0000","last_became_active":"2023-11-28T11:20:48.524517+0000","last_became_peered":"2023-11-28T11:20:48.524517+0000","last_unstale":"2023-11-28T11:20:49.519030+0000","last_undegraded":"2023-11-28T11:20:49.519030+0000","last_fullsized":"2023-11-28T11:20:49.519030+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:49.137767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520673+0000","last_change":"2023-11-28T11:20:48.013633+0000","last_active":"2023-11-28T11:20:49.520673+0000","last_peered":"2023-11-28T11:20:49.520673+0000","last_clean":"2023-11-28T11:20:49.520673+0000","last_became_active":"2023-11-28T11:20:48.013140+0000","last_became_peered":"2023-11-28T11:20:48.013140+0000","last_unstale":"2023-11-28T11:20:49.520673+0000","last_undegraded":"2023-11-28T11:20:49.520673+0000","last_fullsized":"2023-11-28T11:20:49.520673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:23.875674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766264+0000","last_change":"2023-11-28T11:20:48.538976+0000","last_active":"2023-11-28T11:20:49.766264+0000","last_peered":"2023-11-28T11:20:49.766264+0000","last_clean":"2023-11-28T11:20:49.766264+0000","last_became_active":"2023-11-28T11:20:48.536157+0000","last_became_peered":"2023-11-28T11:20:48.536157+0000","last_unstale":"2023-11-28T11:20:49.766264+0000","last_undegraded":"2023-11-28T11:20:49.766264+0000","last_fullsized":"2023-11-28T11:20:49.766264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:14.555768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765764+0000","last_change":"2023-11-28T11:20:48.028182+0000","last_active":"2023-11-28T11:20:49.765764+0000","last_peered":"2023-11-28T11:20:49.765764+0000","last_clean":"2023-11-28T11:20:49.765764+0000","last_became_active":"2023-11-28T11:20:48.027849+0000","last_became_peered":"2023-11-28T11:20:48.027849+0000","last_unstale":"2023-11-28T11:20:49.765764+0000","last_undegraded":"2023-11-28T11:20:49.765764+0000","last_fullsized":"2023-11-28T11:20:49.765764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:40:47.865221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520548+0000","last_change":"2023-11-28T11:20:48.525477+0000","last_active":"2023-11-28T11:20:49.520548+0000","last_peered":"2023-11-28T11:20:49.520548+0000","last_clean":"2023-11-28T11:20:49.520548+0000","last_became_active":"2023-11-28T11:20:48.523568+0000","last_became_peered":"2023-11-28T11:20:48.523568+0000","last_unstale":"2023-11-28T11:20:49.520548+0000","last_undegraded":"2023-11-28T11:20:49.520548+0000","last_fullsized":"2023-11-28T11:20:49.520548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:23.089940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765029+0000","last_change":"2023-11-28T11:20:48.022224+0000","last_active":"2023-11-28T11:20:49.765029+0000","last_peered":"2023-11-28T11:20:49.765029+0000","last_clean":"2023-11-28T11:20:49.765029+0000","last_became_active":"2023-11-28T11:20:48.021901+0000","last_became_peered":"2023-11-28T11:20:48.021901+0000","last_unstale":"2023-11-28T11:20:49.765029+0000","last_undegraded":"2023-11-28T11:20:49.765029+0000","last_fullsized":"2023-11-28T11:20:49.765029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:43.709417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519722+0000","last_change":"2023-11-28T11:20:48.544765+0000","last_active":"2023-11-28T11:20:49.519722+0000","last_peered":"2023-11-28T11:20:49.519722+0000","last_clean":"2023-11-28T11:20:49.519722+0000","last_became_active":"2023-11-28T11:20:48.544501+0000","last_became_peered":"2023-11-28T11:20:48.544501+0000","last_unstale":"2023-11-28T11:20:49.519722+0000","last_undegraded":"2023-11-28T11:20:49.519722+0000","last_fullsized":"2023-11-28T11:20:49.519722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:54.913676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766289+0000","last_change":"2023-11-28T11:20:48.027969+0000","last_active":"2023-11-28T11:20:49.766289+0000","last_peered":"2023-11-28T11:20:49.766289+0000","last_clean":"2023-11-28T11:20:49.766289+0000","last_became_active":"2023-11-28T11:20:48.027627+0000","last_became_peered":"2023-11-28T11:20:48.027627+0000","last_unstale":"2023-11-28T11:20:49.766289+0000","last_undegraded":"2023-11-28T11:20:49.766289+0000","last_fullsized":"2023-11-28T11:20:49.766289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:25:37.461790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519733+0000","last_change":"2023-11-28T11:20:48.523258+0000","last_active":"2023-11-28T11:20:49.519733+0000","last_peered":"2023-11-28T11:20:49.519733+0000","last_clean":"2023-11-28T11:20:49.519733+0000","last_became_active":"2023-11-28T11:20:48.522798+0000","last_became_peered":"2023-11-28T11:20:48.522798+0000","last_unstale":"2023-11-28T11:20:49.519733+0000","last_undegraded":"2023-11-28T11:20:49.519733+0000","last_fullsized":"2023-11-28T11:20:49.519733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:59:53.377101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519810+0000","last_change":"2023-11-28T11:20:48.025831+0000","last_active":"2023-11-28T11:20:49.519810+0000","last_peered":"2023-11-28T11:20:49.519810+0000","last_clean":"2023-11-28T11:20:49.519810+0000","last_became_active":"2023-11-28T11:20:48.025483+0000","last_became_peered":"2023-11-28T11:20:48.025483+0000","last_unstale":"2023-11-28T11:20:49.519810+0000","last_undegraded":"2023-11-28T11:20:49.519810+0000","last_fullsized":"2023-11-28T11:20:49.519810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:27.256053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765678+0000","last_change":"2023-11-28T11:20:48.537610+0000","last_active":"2023-11-28T11:20:49.765678+0000","last_peered":"2023-11-28T11:20:49.765678+0000","last_clean":"2023-11-28T11:20:49.765678+0000","last_became_active":"2023-11-28T11:20:48.536959+0000","last_became_peered":"2023-11-28T11:20:48.536959+0000","last_unstale":"2023-11-28T11:20:49.765678+0000","last_undegraded":"2023-11-28T11:20:49.765678+0000","last_fullsized":"2023-11-28T11:20:49.765678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:53.092247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765384+0000","last_change":"2023-11-28T11:20:48.026660+0000","last_active":"2023-11-28T11:20:49.765384+0000","last_peered":"2023-11-28T11:20:49.765384+0000","last_clean":"2023-11-28T11:20:49.765384+0000","last_became_active":"2023-11-28T11:20:48.024097+0000","last_became_peered":"2023-11-28T11:20:48.024097+0000","last_unstale":"2023-11-28T11:20:49.765384+0000","last_undegraded":"2023-11-28T11:20:49.765384+0000","last_fullsized":"2023-11-28T11:20:49.765384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:21.570329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764852+0000","last_change":"2023-11-28T11:20:48.551384+0000","last_active":"2023-11-28T11:20:49.764852+0000","last_peered":"2023-11-28T11:20:49.764852+0000","last_clean":"2023-11-28T11:20:49.764852+0000","last_became_active":"2023-11-28T11:20:48.550929+0000","last_became_peered":"2023-11-28T11:20:48.550929+0000","last_unstale":"2023-11-28T11:20:49.764852+0000","last_undegraded":"2023-11-28T11:20:49.764852+0000","last_fullsized":"2023-11-28T11:20:49.764852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:17.109714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520142+0000","last_change":"2023-11-28T11:20:48.020231+0000","last_active":"2023-11-28T11:20:49.520142+0000","last_peered":"2023-11-28T11:20:49.520142+0000","last_clean":"2023-11-28T11:20:49.520142+0000","last_became_active":"2023-11-28T11:20:48.019948+0000","last_became_peered":"2023-11-28T11:20:48.019948+0000","last_unstale":"2023-11-28T11:20:49.520142+0000","last_undegraded":"2023-11-28T11:20:49.520142+0000","last_fullsized":"2023-11-28T11:20:49.520142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:40.847246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738473+0000","last_change":"2023-11-28T11:20:48.529292+0000","last_active":"2023-11-28T11:20:49.738473+0000","last_peered":"2023-11-28T11:20:49.738473+0000","last_clean":"2023-11-28T11:20:49.738473+0000","last_became_active":"2023-11-28T11:20:48.528852+0000","last_became_peered":"2023-11-28T11:20:48.528852+0000","last_unstale":"2023-11-28T11:20:49.738473+0000","last_undegraded":"2023-11-28T11:20:49.738473+0000","last_fullsized":"2023-11-28T11:20:49.738473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:31.033368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520885+0000","last_change":"2023-11-28T11:20:48.029979+0000","last_active":"2023-11-28T11:20:49.520885+0000","last_peered":"2023-11-28T11:20:49.520885+0000","last_clean":"2023-11-28T11:20:49.520885+0000","last_became_active":"2023-11-28T11:20:48.029704+0000","last_became_peered":"2023-11-28T11:20:48.029704+0000","last_unstale":"2023-11-28T11:20:49.520885+0000","last_undegraded":"2023-11-28T11:20:49.520885+0000","last_fullsized":"2023-11-28T11:20:49.520885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:02.883308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765378+0000","last_change":"2023-11-28T11:20:48.540593+0000","last_active":"2023-11-28T11:20:49.765378+0000","last_peered":"2023-11-28T11:20:49.765378+0000","last_clean":"2023-11-28T11:20:49.765378+0000","last_became_active":"2023-11-28T11:20:48.539911+0000","last_became_peered":"2023-11-28T11:20:48.539911+0000","last_unstale":"2023-11-28T11:20:49.765378+0000","last_undegraded":"2023-11-28T11:20:49.765378+0000","last_fullsized":"2023-11-28T11:20:49.765378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:47.633995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520854+0000","last_change":"2023-11-28T11:20:48.024865+0000","last_active":"2023-11-28T11:20:49.520854+0000","last_peered":"2023-11-28T11:20:49.520854+0000","last_clean":"2023-11-28T11:20:49.520854+0000","last_became_active":"2023-11-28T11:20:48.024229+0000","last_became_peered":"2023-11-28T11:20:48.024229+0000","last_unstale":"2023-11-28T11:20:49.520854+0000","last_undegraded":"2023-11-28T11:20:49.520854+0000","last_fullsized":"2023-11-28T11:20:49.520854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:23.122350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520857+0000","last_change":"2023-11-28T11:20:48.526408+0000","last_active":"2023-11-28T11:20:49.520857+0000","last_peered":"2023-11-28T11:20:49.520857+0000","last_clean":"2023-11-28T11:20:49.520857+0000","last_became_active":"2023-11-28T11:20:48.526142+0000","last_became_peered":"2023-11-28T11:20:48.526142+0000","last_unstale":"2023-11-28T11:20:49.520857+0000","last_undegraded":"2023-11-28T11:20:49.520857+0000","last_fullsized":"2023-11-28T11:20:49.520857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:16:43.336081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738644+0000","last_change":"2023-11-28T11:20:48.024784+0000","last_active":"2023-11-28T11:20:49.738644+0000","last_peered":"2023-11-28T11:20:49.738644+0000","last_clean":"2023-11-28T11:20:49.738644+0000","last_became_active":"2023-11-28T11:20:48.024517+0000","last_became_peered":"2023-11-28T11:20:48.024517+0000","last_unstale":"2023-11-28T11:20:49.738644+0000","last_undegraded":"2023-11-28T11:20:49.738644+0000","last_fullsized":"2023-11-28T11:20:49.738644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:12.341011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766430+0000","last_change":"2023-11-28T11:20:48.533571+0000","last_active":"2023-11-28T11:20:49.766430+0000","last_peered":"2023-11-28T11:20:49.766430+0000","last_clean":"2023-11-28T11:20:49.766430+0000","last_became_active":"2023-11-28T11:20:48.533197+0000","last_became_peered":"2023-11-28T11:20:48.533197+0000","last_unstale":"2023-11-28T11:20:49.766430+0000","last_undegraded":"2023-11-28T11:20:49.766430+0000","last_fullsized":"2023-11-28T11:20:49.766430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:24.882063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764869+0000","last_change":"2023-11-28T11:20:48.022152+0000","last_active":"2023-11-28T11:20:49.764869+0000","last_peered":"2023-11-28T11:20:49.764869+0000","last_clean":"2023-11-28T11:20:49.764869+0000","last_became_active":"2023-11-28T11:20:48.021851+0000","last_became_peered":"2023-11-28T11:20:48.021851+0000","last_unstale":"2023-11-28T11:20:49.764869+0000","last_undegraded":"2023-11-28T11:20:49.764869+0000","last_fullsized":"2023-11-28T11:20:49.764869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:59.717672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765344+0000","last_change":"2023-11-28T11:20:48.533747+0000","last_active":"2023-11-28T11:20:49.765344+0000","last_peered":"2023-11-28T11:20:49.765344+0000","last_clean":"2023-11-28T11:20:49.765344+0000","last_became_active":"2023-11-28T11:20:48.533423+0000","last_became_peered":"2023-11-28T11:20:48.533423+0000","last_unstale":"2023-11-28T11:20:49.765344+0000","last_undegraded":"2023-11-28T11:20:49.765344+0000","last_fullsized":"2023-11-28T11:20:49.765344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:11.339425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766287+0000","last_change":"2023-11-28T11:20:48.023083+0000","last_active":"2023-11-28T11:20:49.766287+0000","last_peered":"2023-11-28T11:20:49.766287+0000","last_clean":"2023-11-28T11:20:49.766287+0000","last_became_active":"2023-11-28T11:20:48.022757+0000","last_became_peered":"2023-11-28T11:20:48.022757+0000","last_unstale":"2023-11-28T11:20:49.766287+0000","last_undegraded":"2023-11-28T11:20:49.766287+0000","last_fullsized":"2023-11-28T11:20:49.766287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:51.000937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520831+0000","last_change":"2023-11-28T11:20:48.552702+0000","last_active":"2023-11-28T11:20:49.520831+0000","last_peered":"2023-11-28T11:20:49.520831+0000","last_clean":"2023-11-28T11:20:49.520831+0000","last_became_active":"2023-11-28T11:20:48.552233+0000","last_became_peered":"2023-11-28T11:20:48.552233+0000","last_unstale":"2023-11-28T11:20:49.520831+0000","last_undegraded":"2023-11-28T11:20:49.520831+0000","last_fullsized":"2023-11-28T11:20:49.520831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:58.059136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518827+0000","last_change":"2023-11-28T11:20:48.009911+0000","last_active":"2023-11-28T11:20:49.518827+0000","last_peered":"2023-11-28T11:20:49.518827+0000","last_clean":"2023-11-28T11:20:49.518827+0000","last_became_active":"2023-11-28T11:20:48.008934+0000","last_became_peered":"2023-11-28T11:20:48.008934+0000","last_unstale":"2023-11-28T11:20:49.518827+0000","last_undegraded":"2023-11-28T11:20:49.518827+0000","last_fullsized":"2023-11-28T11:20:49.518827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.907287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764577+0000","last_change":"2023-11-28T11:20:48.530423+0000","last_active":"2023-11-28T11:20:49.764577+0000","last_peered":"2023-11-28T11:20:49.764577+0000","last_clean":"2023-11-28T11:20:49.764577+0000","last_became_active":"2023-11-28T11:20:48.530145+0000","last_became_peered":"2023-11-28T11:20:48.530145+0000","last_unstale":"2023-11-28T11:20:49.764577+0000","last_undegraded":"2023-11-28T11:20:49.764577+0000","last_fullsized":"2023-11-28T11:20:49.764577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:01.527043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518720+0000","last_change":"2023-11-28T11:20:48.012947+0000","last_active":"2023-11-28T11:20:49.518720+0000","last_peered":"2023-11-28T11:20:49.518720+0000","last_clean":"2023-11-28T11:20:49.518720+0000","last_became_active":"2023-11-28T11:20:48.012655+0000","last_became_peered":"2023-11-28T11:20:48.012655+0000","last_unstale":"2023-11-28T11:20:49.518720+0000","last_undegraded":"2023-11-28T11:20:49.518720+0000","last_fullsized":"2023-11-28T11:20:49.518720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:43.828885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765075+0000","last_change":"2023-11-28T11:20:48.534132+0000","last_active":"2023-11-28T11:20:49.765075+0000","last_peered":"2023-11-28T11:20:49.765075+0000","last_clean":"2023-11-28T11:20:49.765075+0000","last_became_active":"2023-11-28T11:20:48.533886+0000","last_became_peered":"2023-11-28T11:20:48.533886+0000","last_unstale":"2023-11-28T11:20:49.765075+0000","last_undegraded":"2023-11-28T11:20:49.765075+0000","last_fullsized":"2023-11-28T11:20:49.765075+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:32.103922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.764327+0000","last_change":"2023-11-28T11:20:48.021093+0000","last_active":"2023-11-28T11:20:49.764327+0000","last_peered":"2023-11-28T11:20:49.764327+0000","last_clean":"2023-11-28T11:20:49.764327+0000","last_became_active":"2023-11-28T11:20:48.020810+0000","last_became_peered":"2023-11-28T11:20:48.020810+0000","last_unstale":"2023-11-28T11:20:49.764327+0000","last_undegraded":"2023-11-28T11:20:49.764327+0000","last_fullsized":"2023-11-28T11:20:49.764327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:40.847246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765896+0000","last_change":"2023-11-28T11:20:48.552219+0000","last_active":"2023-11-28T11:20:49.765896+0000","last_peered":"2023-11-28T11:20:49.765896+0000","last_clean":"2023-11-28T11:20:49.765896+0000","last_became_active":"2023-11-28T11:20:48.551927+0000","last_became_peered":"2023-11-28T11:20:48.551927+0000","last_unstale":"2023-11-28T11:20:49.765896+0000","last_undegraded":"2023-11-28T11:20:49.765896+0000","last_fullsized":"2023-11-28T11:20:49.765896+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:55.264867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520794+0000","last_change":"2023-11-28T11:20:48.025820+0000","last_active":"2023-11-28T11:20:49.520794+0000","last_peered":"2023-11-28T11:20:49.520794+0000","last_clean":"2023-11-28T11:20:49.520794+0000","last_became_active":"2023-11-28T11:20:48.025349+0000","last_became_peered":"2023-11-28T11:20:48.025349+0000","last_unstale":"2023-11-28T11:20:49.520794+0000","last_undegraded":"2023-11-28T11:20:49.520794+0000","last_fullsized":"2023-11-28T11:20:49.520794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:46.279307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.519838+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:20:48.519838+0000","last_peered":"2023-11-28T11:20:48.519838+0000","last_clean":"2023-11-28T11:20:48.519838+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:20:48.519838+0000","last_undegraded":"2023-11-28T11:20:48.519838+0000","last_fullsized":"2023-11-28T11:20:48.519838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765729+0000","last_change":"2023-11-28T11:20:48.536656+0000","last_active":"2023-11-28T11:20:49.765729+0000","last_peered":"2023-11-28T11:20:49.765729+0000","last_clean":"2023-11-28T11:20:49.765729+0000","last_became_active":"2023-11-28T11:20:48.532636+0000","last_became_peered":"2023-11-28T11:20:48.532636+0000","last_unstale":"2023-11-28T11:20:49.765729+0000","last_undegraded":"2023-11-28T11:20:49.765729+0000","last_fullsized":"2023-11-28T11:20:49.765729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:29.029468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738243+0000","last_change":"2023-11-28T11:20:48.019569+0000","last_active":"2023-11-28T11:20:49.738243+0000","last_peered":"2023-11-28T11:20:49.738243+0000","last_clean":"2023-11-28T11:20:49.738243+0000","last_became_active":"2023-11-28T11:20:48.019249+0000","last_became_peered":"2023-11-28T11:20:48.019249+0000","last_unstale":"2023-11-28T11:20:49.738243+0000","last_undegraded":"2023-11-28T11:20:49.738243+0000","last_fullsized":"2023-11-28T11:20:49.738243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:41.529820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.518399+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:20:48.518399+0000","last_peered":"2023-11-28T11:20:48.518399+0000","last_clean":"2023-11-28T11:20:48.518399+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:20:48.518399+0000","last_undegraded":"2023-11-28T11:20:48.518399+0000","last_fullsized":"2023-11-28T11:20:48.518399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520608+0000","last_change":"2023-11-28T11:20:48.530075+0000","last_active":"2023-11-28T11:20:49.520608+0000","last_peered":"2023-11-28T11:20:49.520608+0000","last_clean":"2023-11-28T11:20:49.520608+0000","last_became_active":"2023-11-28T11:20:48.529770+0000","last_became_peered":"2023-11-28T11:20:48.529770+0000","last_unstale":"2023-11-28T11:20:49.520608+0000","last_undegraded":"2023-11-28T11:20:49.520608+0000","last_fullsized":"2023-11-28T11:20:49.520608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:59.748308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.519902+0000","last_change":"2023-11-28T11:20:48.026008+0000","last_active":"2023-11-28T11:20:49.519902+0000","last_peered":"2023-11-28T11:20:49.519902+0000","last_clean":"2023-11-28T11:20:49.519902+0000","last_became_active":"2023-11-28T11:20:48.025671+0000","last_became_peered":"2023-11-28T11:20:48.025671+0000","last_unstale":"2023-11-28T11:20:49.519902+0000","last_undegraded":"2023-11-28T11:20:49.519902+0000","last_fullsized":"2023-11-28T11:20:49.519902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:37:43.991234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.529670+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:20:48.529670+0000","last_peered":"2023-11-28T11:20:48.529670+0000","last_clean":"2023-11-28T11:20:48.529670+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:20:48.529670+0000","last_undegraded":"2023-11-28T11:20:48.529670+0000","last_fullsized":"2023-11-28T11:20:48.529670+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765991+0000","last_change":"2023-11-28T11:20:48.533762+0000","last_active":"2023-11-28T11:20:49.765991+0000","last_peered":"2023-11-28T11:20:49.765991+0000","last_clean":"2023-11-28T11:20:49.765991+0000","last_became_active":"2023-11-28T11:20:48.533266+0000","last_became_peered":"2023-11-28T11:20:48.533266+0000","last_unstale":"2023-11-28T11:20:49.765991+0000","last_undegraded":"2023-11-28T11:20:49.765991+0000","last_fullsized":"2023-11-28T11:20:49.765991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:23.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518585+0000","last_change":"2023-11-28T11:20:48.010534+0000","last_active":"2023-11-28T11:20:49.518585+0000","last_peered":"2023-11-28T11:20:49.518585+0000","last_clean":"2023-11-28T11:20:49.518585+0000","last_became_active":"2023-11-28T11:20:48.010273+0000","last_became_peered":"2023-11-28T11:20:48.010273+0000","last_unstale":"2023-11-28T11:20:49.518585+0000","last_undegraded":"2023-11-28T11:20:49.518585+0000","last_fullsized":"2023-11-28T11:20:49.518585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:35.795225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.520108+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:20:48.520108+0000","last_peered":"2023-11-28T11:20:48.520108+0000","last_clean":"2023-11-28T11:20:48.520108+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:20:48.520108+0000","last_undegraded":"2023-11-28T11:20:48.520108+0000","last_fullsized":"2023-11-28T11:20:48.520108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.526547+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:20:48.526547+0000","last_peered":"2023-11-28T11:20:48.526547+0000","last_clean":"2023-11-28T11:20:48.526547+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:20:48.526547+0000","last_undegraded":"2023-11-28T11:20:48.526547+0000","last_fullsized":"2023-11-28T11:20:48.526547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738308+0000","last_change":"2023-11-28T11:20:48.533511+0000","last_active":"2023-11-28T11:20:49.738308+0000","last_peered":"2023-11-28T11:20:49.738308+0000","last_clean":"2023-11-28T11:20:49.738308+0000","last_became_active":"2023-11-28T11:20:48.532305+0000","last_became_peered":"2023-11-28T11:20:48.532305+0000","last_unstale":"2023-11-28T11:20:49.738308+0000","last_undegraded":"2023-11-28T11:20:49.738308+0000","last_fullsized":"2023-11-28T11:20:49.738308+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:04.028115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.738455+0000","last_change":"2023-11-28T11:20:48.021173+0000","last_active":"2023-11-28T11:20:49.738455+0000","last_peered":"2023-11-28T11:20:49.738455+0000","last_clean":"2023-11-28T11:20:49.738455+0000","last_became_active":"2023-11-28T11:20:48.020868+0000","last_became_peered":"2023-11-28T11:20:48.020868+0000","last_unstale":"2023-11-28T11:20:49.738455+0000","last_undegraded":"2023-11-28T11:20:49.738455+0000","last_fullsized":"2023-11-28T11:20:49.738455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:15.620609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.526447+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:20:48.526447+0000","last_peered":"2023-11-28T11:20:48.526447+0000","last_clean":"2023-11-28T11:20:48.526447+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:20:48.526447+0000","last_undegraded":"2023-11-28T11:20:48.526447+0000","last_fullsized":"2023-11-28T11:20:48.526447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766246+0000","last_change":"2023-11-28T11:20:48.536502+0000","last_active":"2023-11-28T11:20:49.766246+0000","last_peered":"2023-11-28T11:20:49.766246+0000","last_clean":"2023-11-28T11:20:49.766246+0000","last_became_active":"2023-11-28T11:20:48.535985+0000","last_became_peered":"2023-11-28T11:20:48.535985+0000","last_unstale":"2023-11-28T11:20:49.766246+0000","last_undegraded":"2023-11-28T11:20:49.766246+0000","last_fullsized":"2023-11-28T11:20:49.766246+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:40:48.871611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766125+0000","last_change":"2023-11-28T11:20:48.027672+0000","last_active":"2023-11-28T11:20:49.766125+0000","last_peered":"2023-11-28T11:20:49.766125+0000","last_clean":"2023-11-28T11:20:49.766125+0000","last_became_active":"2023-11-28T11:20:48.024198+0000","last_became_peered":"2023-11-28T11:20:48.024198+0000","last_unstale":"2023-11-28T11:20:49.766125+0000","last_undegraded":"2023-11-28T11:20:49.766125+0000","last_fullsized":"2023-11-28T11:20:49.766125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:37:49.035634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.532273+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:20:48.532273+0000","last_peered":"2023-11-28T11:20:48.532273+0000","last_clean":"2023-11-28T11:20:48.532273+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:20:48.532273+0000","last_undegraded":"2023-11-28T11:20:48.532273+0000","last_fullsized":"2023-11-28T11:20:48.532273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.520569+0000","last_change":"2023-11-28T11:20:48.534560+0000","last_active":"2023-11-28T11:20:49.520569+0000","last_peered":"2023-11-28T11:20:49.520569+0000","last_clean":"2023-11-28T11:20:49.520569+0000","last_became_active":"2023-11-28T11:20:48.534258+0000","last_became_peered":"2023-11-28T11:20:48.534258+0000","last_unstale":"2023-11-28T11:20:49.520569+0000","last_undegraded":"2023-11-28T11:20:49.520569+0000","last_fullsized":"2023-11-28T11:20:49.520569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:01.527043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.765682+0000","last_change":"2023-11-28T11:20:48.028866+0000","last_active":"2023-11-28T11:20:49.765682+0000","last_peered":"2023-11-28T11:20:49.765682+0000","last_clean":"2023-11-28T11:20:49.765682+0000","last_became_active":"2023-11-28T11:20:48.028486+0000","last_became_peered":"2023-11-28T11:20:48.028486+0000","last_unstale":"2023-11-28T11:20:49.765682+0000","last_undegraded":"2023-11-28T11:20:49.765682+0000","last_fullsized":"2023-11-28T11:20:49.765682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:48.674225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.520821+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:20:48.520821+0000","last_peered":"2023-11-28T11:20:48.520821+0000","last_clean":"2023-11-28T11:20:48.520821+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:20:48.520821+0000","last_undegraded":"2023-11-28T11:20:48.520821+0000","last_fullsized":"2023-11-28T11:20:48.520821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.766501+0000","last_change":"2023-11-28T11:20:48.526496+0000","last_active":"2023-11-28T11:20:49.766501+0000","last_peered":"2023-11-28T11:20:49.766501+0000","last_clean":"2023-11-28T11:20:49.766501+0000","last_became_active":"2023-11-28T11:20:48.526203+0000","last_became_peered":"2023-11-28T11:20:48.526203+0000","last_unstale":"2023-11-28T11:20:49.766501+0000","last_undegraded":"2023-11-28T11:20:49.766501+0000","last_fullsized":"2023-11-28T11:20:49.766501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:47.995404+0000","last_clean_scrub_stamp":"2023-11-28T11:20:47.995404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:44.715807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T11:20:49.518899+0000","last_change":"2023-11-28T11:20:48.010177+0000","last_active":"2023-11-28T11:20:49.518899+0000","last_peered":"2023-11-28T11:20:49.518899+0000","last_clean":"2023-11-28T11:20:49.518899+0000","last_became_active":"2023-11-28T11:20:48.009920+0000","last_became_peered":"2023-11-28T11:20:48.009920+0000","last_unstale":"2023-11-28T11:20:49.518899+0000","last_undegraded":"2023-11-28T11:20:49.518899+0000","last_fullsized":"2023-11-28T11:20:49.518899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:46.989015+0000","last_clean_scrub_stamp":"2023-11-28T11:20:46.989015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:16.103324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:20:48.528559+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:20:48.528559+0000","last_peered":"2023-11-28T11:20:48.528559+0000","last_clean":"2023-11-28T11:20:48.528559+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:20:48.528559+0000","last_undegraded":"2023-11-28T11:20:48.528559+0000","last_fullsized":"2023-11-28T11:20:48.528559+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14884,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356956,"statfs":{"total":96636764160,"available":96621522944,"internally_reserved":0,"allocated":1339392,"data_stored":911821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.62,"15min":0.626},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88300000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81599999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.83299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84199999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84199999999999997,"5min":1.018,"15min":1.018},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83699999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.745}]},{"osd":4,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.87,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83599999999999997,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.79500000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13396,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358444,"statfs":{"total":96636764160,"available":96623046656,"internally_reserved":0,"allocated":733184,"data_stored":320727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.071,"5min":2.23,"15min":2.23},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78000000000000003,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95899999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89300000000000002,"5min":1.147,"15min":1.147},"last":0.77500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.66800000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66900000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93500000000000005,"5min":1.224,"15min":1.224},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.57899999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0029999999999999,"5min":1.627,"15min":1.627},"last":0.80200000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94199999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85999999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96499999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.876},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.90200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13608,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358232,"statfs":{"total":96636764160,"available":96622829568,"internally_reserved":0,"allocated":753664,"data_stored":322025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40899999999999997,"15min":0.41699999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.44600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.314,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53100000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59699999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61299999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.40899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.496},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.46700000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":1.181,"15min":1.181},"last":0.66800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.84999999999999998,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.749,"15min":0.77700000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.774,"15min":1.774},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.56899999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.92700000000000005,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76200000000000001,"15min":0.77700000000000002},"min":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.93000000000000005,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70199999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14672,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":13376,"kb_avail":94357168,"statfs":{"total":96636764160,"available":96621740032,"internally_reserved":0,"allocated":1318912,"data_stored":910042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13697024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47399999999999998,"15min":0.48599999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.60199999999999998,"5min":1.052,"15min":1.052},"last":0.311},{"interface":"front","average":{"1min":0.501,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.028,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.35199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55700000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.753}]},{"osd":5,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79600000000000004,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1879999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76400000000000001,"15min":0.76200000000000001},"min":{"1min":0.64500000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.173,"5min":1.458,"15min":1.458},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.56799999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.66300000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.591,"15min":1.591},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.64900000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.629,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.214,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81599999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13520,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94358320,"statfs":{"total":96636764160,"available":96622919680,"internally_reserved":0,"allocated":729088,"data_stored":320693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.56200000000000006,"15min":0.62},"min":{"1min":0.29399999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.64100000000000001,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54500000000000004,"15min":0.57299999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63300000000000001,"5min":1.04,"15min":1.04},"last":0.48399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.55900000000000005,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.67400000000000004,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.57199999999999995,"15min":0.61499999999999999},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.97299999999999998,"5min":4.093,"15min":4.093},"last":0.442},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.72899999999999998,"15min":0.80700000000000005},"min":{"1min":0.45200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.99199999999999999,"5min":3.956,"15min":3.956},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78400000000000003,"15min":0.82599999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94299999999999995,"5min":2.028,"15min":2.028},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.52800000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.878,"5min":1.147,"15min":1.147},"last":0.68999999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.88400000000000001,"15min":0.96899999999999997},"min":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.022,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.90200000000000002,"15min":0.96999999999999997},"min":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.004,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.73299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.63500000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.96199999999999997,"15min":1.0429999999999999},"min":{"1min":0.61699999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0409999999999999,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.91900000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82799999999999996,"15min":0.86199999999999999},"min":{"1min":0.622,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95599999999999996,"5min":1.272,"15min":1.272},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.008},"min":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96899999999999997,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.77000000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13840,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94358000,"statfs":{"total":96636764160,"available":96622592000,"internally_reserved":0,"allocated":729088,"data_stored":320693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.51500000000000001,"15min":0.55200000000000005},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.32100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.58899999999999997,"15min":0.63800000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":3.004,"15min":3.004},"last":0.44800000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63100000000000001,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66400000000000003,"5min":3.274,"15min":3.274},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48099999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.02,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94999999999999996,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.006,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.998,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81100000000000005,"15min":0.80700000000000005},"min":{"1min":0.67700000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.59199999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82599999999999996,"15min":0.83399999999999996},"min":{"1min":0.65900000000000003,"5min":0.622,"15min":0.622},"max":{"1min":0.98299999999999998,"5min":1.099,"15min":1.099},"last":0.70799999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13724,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358116,"statfs":{"total":96636764160,"available":96622710784,"internally_reserved":0,"allocated":741376,"data_stored":322241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.61399999999999999,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85599999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.495},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.53600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.46100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77600000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.59499999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.47599999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.752},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.623,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.873,"5min":1.847,"15min":1.847},"last":0.71799999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94099999999999995,"5min":1.615,"15min":1.615},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":1.127,"15min":1.127},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86799999999999999,"5min":1.089,"15min":1.089},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78000000000000003,"15min":0.79300000000000004},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94099999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84899999999999998,"5min":1.296,"15min":1.296},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89300000000000002,"5min":1.212,"15min":1.212},"last":0.83499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":733184,"data_stored":320715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.65200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.73399999999999999,"15min":0.73299999999999998},"min":{"1min":0.45000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82199999999999995,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":2.661,"15min":2.661},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.97199999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.81200000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.61799999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0349999999999999,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89700000000000002,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":5.375,"15min":5.375},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.47399999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79000000000000004,"15min":0.78000000000000003},"min":{"1min":0.53900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88600000000000001,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.754,"15min":0.78000000000000003},"min":{"1min":0.47999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91400000000000003,"5min":1.333,"15min":1.333},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78300000000000003,"15min":0.77800000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.73799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:20:53.427 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:20:53.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:20:53.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.669+0000 7fea18fff700 1 Processor -- start 2023-11-28T11:20:53.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.669+0000 7fea18fff700 1 -- start start 2023-11-28T11:20:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.670+0000 7fea18fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fea1415c9b0 0x7fea140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.670+0000 7fea18fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140d1950 con 0x7fea140cd7c0 2023-11-28T11:20:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.670+0000 7fea18fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140d24f0 con 0x7fea14161340 2023-11-28T11:20:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.670+0000 7fea18fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140b5800 con 0x7fea1415c9b0 2023-11-28T11:20:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea1259c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fea1415c9b0 0x7fea140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea12d9d700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fea140cd7c0 0x7fea140b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36288/0 (socket says 172.21.15.53:36288) 2023-11-28T11:20:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea12d9d700 1 -- 172.21.15.53:0/3343716786 learned_addr learned my addr 172.21.15.53:0/3343716786 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea1159a700 1 -- 172.21.15.53:0/3343716786 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081325138 0 0) 0x7fea140d1950 con 0x7fea140cd7c0 2023-11-28T11:20:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea1259c700 1 -- 172.21.15.53:0/3343716786 >> v1:172.21.15.157:6789/0 conn(0x7fea14161340 legacy=0x7fea140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea1259c700 1 -- 172.21.15.53:0/3343716786 >> v1:172.21.15.53:6789/0 conn(0x7fea140cd7c0 legacy=0x7fea140b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.671+0000 7fea1259c700 1 -- 172.21.15.53:0/3343716786 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140b5520 con 0x7fea1415c9b0 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea1259c700 1 --2- 172.21.15.53:0/3343716786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1415c9b0 0x7fea140b4bf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fea04009840 tx=0x7fea04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478ddf34484198a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea1159a700 1 -- 172.21.15.53:0/3343716786 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011830 con 0x7fea1415c9b0 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea1159a700 1 -- 172.21.15.53:0/3343716786 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04011990 con 0x7fea1415c9b0 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea1159a700 1 -- 172.21.15.53:0/3343716786 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011d20 con 0x7fea1415c9b0 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea18fff700 1 -- 172.21.15.53:0/3343716786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1415c9b0 msgr2=0x7fea140b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea18fff700 1 --2- 172.21.15.53:0/3343716786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1415c9b0 0x7fea140b4bf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fea04009840 tx=0x7fea04009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea18fff700 1 -- 172.21.15.53:0/3343716786 shutdown_connections 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea18fff700 1 --2- 172.21.15.53:0/3343716786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1415c9b0 0x7fea140b4bf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.672+0000 7fea18fff700 1 -- 172.21.15.53:0/3343716786 >> 172.21.15.53:0/3343716786 conn(0x7fea14072600 msgr2=0x7fea140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.673+0000 7fea18fff700 1 -- 172.21.15.53:0/3343716786 shutdown_connections 2023-11-28T11:20:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.673+0000 7fea18fff700 1 -- 172.21.15.53:0/3343716786 wait complete. 2023-11-28T11:20:53.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.674+0000 7fea18fff700 1 Processor -- start 2023-11-28T11:20:53.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.674+0000 7fea18fff700 1 -- start start 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 0x7fea141572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 0x7fea140c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141e6d90 con 0x7fea140cd7c0 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141e6480 con 0x7fea14161340 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.675+0000 7fea18fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140c8e30 con 0x7fea1415c9b0 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38210/0 (socket says 172.21.15.53:38210) 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 -- 172.21.15.53:0/317086701 learned_addr learned my addr 172.21.15.53:0/317086701 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea11d9b700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 0x7fea141572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea12d9d700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 0x7fea140c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 -- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 msgr2=0x7fea141572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 0x7fea141572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 -- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 msgr2=0x7fea140c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 0x7fea140c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 -- 172.21.15.53:0/317086701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040094a0 con 0x7fea140cd7c0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea11d9b700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 0x7fea141572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea12d9d700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 0x7fea140c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea1259c700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fea04009b90 tx=0x7fea04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.676+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040036a0 con 0x7fea140cd7c0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.677+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0401e070 con 0x7fea140cd7c0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.677+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04003a00 con 0x7fea140cd7c0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.677+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea140d15e0 con 0x7fea140cd7c0 2023-11-28T11:20:53.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.677+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea140d1300 con 0x7fea140cd7c0 2023-11-28T11:20:53.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.678+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea140b5130 con 0x7fea140cd7c0 2023-11-28T11:20:53.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.679+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea0400f050 con 0x7fea140cd7c0 2023-11-28T11:20:53.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.679+0000 7fea02ffd700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 0x7fe9f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.679+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea04086ef0 con 0x7fea140cd7c0 2023-11-28T11:20:53.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.680+0000 7fea11d9b700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 0x7fe9f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.680+0000 7fea11d9b700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 0x7fe9f406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea141e7be0 tx=0x7fea08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.684+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea04053660 con 0x7fea140cd7c0 2023-11-28T11:20:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.965+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fea140745c0 con 0x7fea140cd7c0 2023-11-28T11:20:53.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.969+0000 7fea02ffd700 1 -- 172.21.15.53:0/317086701 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fea040150e0 con 0x7fea140cd7c0 2023-11-28T11:20:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 msgr2=0x7fe9f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 0x7fe9f406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea141e7be0 tx=0x7fea08009040 comp rx=0 tx=0).stop 2023-11-28T11:20:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 msgr2=0x7fea14156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fea04009b90 tx=0x7fea04004750 comp rx=0 tx=0).stop 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 shutdown_connections 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe9f40682f0 0x7fe9f406a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14161340 0x7fea140c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140cd7c0 0x7fea14156d60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 --2- 172.21.15.53:0/317086701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea1415c9b0 0x7fea141572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 >> 172.21.15.53:0/317086701 conn(0x7fea14072600 msgr2=0x7fea14160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 shutdown_connections 2023-11-28T11:20:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:53.972+0000 7fea18fff700 1 -- 172.21.15.53:0/317086701 wait complete. 2023-11-28T11:20:53.976 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-11-28T11:20:53.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:54.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.192+0000 7f48f93ff700 1 Processor -- start 2023-11-28T11:20:54.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.192+0000 7f48f93ff700 1 -- start start 2023-11-28T11:20:54.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.193+0000 7f48f93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48f40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f93ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40d1950 con 0x7f48f415c9b0 2023-11-28T11:20:54.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f93ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40ce140 con 0x7f48f4161340 2023-11-28T11:20:54.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40b5800 con 0x7f48f40cd7c0 2023-11-28T11:20:54.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f2ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f48f415c9b0 0x7f48f40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36296/0 (socket says 172.21.15.53:36296) 2023-11-28T11:20:54.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f2ffd700 1 -- 172.21.15.53:0/3491332306 learned_addr learned my addr 172.21.15.53:0/3491332306 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:54.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48f40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962705570 0 0) 0x7f48f40d1950 con 0x7f48f415c9b0 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e00035b0 con 0x7f48f415c9b0 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81632468 0 0) 0x7f48f40ce140 con 0x7f48f4161340 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.194+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f40d1950 con 0x7f48f4161340 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520318863 0 0) 0x7f48e00035b0 con 0x7f48f415c9b0 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 msgr2=0x7f48f40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48f40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 >> v1:172.21.15.157:6789/0 conn(0x7f48f4161340 legacy=0x7f48f40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40d24f0 con 0x7f48f415c9b0 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f48e4001670 con 0x7f48f415c9b0 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48f40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:54.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 >> v1:172.21.15.53:6789/0 conn(0x7f48f415c9b0 legacy=0x7f48f40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e0003bc0 0x7f48e0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e0004d90 0x7f48e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40b5800 con 0x7f48e0004d90 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e000bb40 con 0x7f48e0003bc0 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e000bd00 con 0x7f48f40cd7c0 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e0004d90 0x7f48e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f2ffd700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e0003bc0 0x7f48e0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48ebfff700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 -- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 msgr2=0x7f48e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 -- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e0003bc0 msgr2=0x7f48e0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e0003bc0 0x7f48e0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48f37fe700 1 -- 172.21.15.53:0/3491332306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009a70 con 0x7f48e0004d90 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.195+0000 7f48ebfff700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48f40cd7c0 0x7f48e0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f37fe700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e0004d90 0x7f48e0005200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e350 tx=0x7f48e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d111e1ea15c97295 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00edf0 con 0x7f48e0004d90 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00f040 con 0x7f48e0004d90 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f27fc700 1 -- 172.21.15.53:0/3491332306 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0146a0 con 0x7f48e0004d90 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f93ff700 1 -- 172.21.15.53:0/3491332306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e0004d90 msgr2=0x7f48e0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f93ff700 1 --2- 172.21.15.53:0/3491332306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e0004d90 0x7f48e0005200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e350 tx=0x7f48e0005880 comp rx=0 tx=0).stop 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f37fe700 1 -- 172.21.15.53:0/3491332306 reap_dead start 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f93ff700 1 -- 172.21.15.53:0/3491332306 shutdown_connections 2023-11-28T11:20:54.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.196+0000 7f48f93ff700 1 -- 172.21.15.53:0/3491332306 >> 172.21.15.53:0/3491332306 conn(0x7f48f4072600 msgr2=0x7f48f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.197+0000 7f48f93ff700 1 -- 172.21.15.53:0/3491332306 shutdown_connections 2023-11-28T11:20:54.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.197+0000 7f48f93ff700 1 -- 172.21.15.53:0/3491332306 wait complete. 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.198+0000 7f48f93ff700 1 Processor -- start 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 -- start start 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48e0004d90 0x7f48f40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f415f290 0x7f48f4149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c71e0 con 0x7f48f415e3c0 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f415ed70 con 0x7f48f415f290 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.199+0000 7f48f93ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4149d90 con 0x7f48e0004d90 2023-11-28T11:20:54.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38228/0 (socket says 172.21.15.53:38228) 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 -- 172.21.15.53:0/1451314728 learned_addr learned my addr 172.21.15.53:0/1451314728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f2ffd700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48e0004d90 0x7f48f40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48ebfff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f415f290 0x7f48f4149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 -- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48e0004d90 msgr2=0x7f48f40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48e0004d90 0x7f48f40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 -- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f415f290 msgr2=0x7f48f4149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f415f290 0x7f48f4149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 -- 172.21.15.53:0/1451314728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40d1300 con 0x7f48f415e3c0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.200+0000 7f48f37fe700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e940 tx=0x7f48ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.201+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec007000 con 0x7f48f415e3c0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.201+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec004430 con 0x7f48f415e3c0 2023-11-28T11:20:54.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.201+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec004820 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.201+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40b5520 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.201+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40d15e0 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.202+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40ce140 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.203+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48ec02e070 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.204+0000 7f48eb7fe700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 0x7f48d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.204+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48ec087750 con 0x7f48f415e3c0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.204+0000 7f48f2ffd700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 0x7f48d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.205+0000 7f48f2ffd700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 0x7f48d406a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48e4001eb0 tx=0x7f48e4000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.208+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48ec053ec0 con 0x7f48f415e3c0 2023-11-28T11:20:54.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.454+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48f40745c0 con 0x7f48f415e3c0 2023-11-28T11:20:54.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.455+0000 7f48eb7fe700 1 -- 172.21.15.53:0/1451314728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f48ec0574e0 con 0x7f48f415e3c0 2023-11-28T11:20:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:54.457 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:20:51.523382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:20:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 msgr2=0x7f48d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 0x7f48d406a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48e4001eb0 tx=0x7f48e4000f80 comp rx=0 tx=0).stop 2023-11-28T11:20:54.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 msgr2=0x7f48f415e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e940 tx=0x7f48ec009750 comp rx=0 tx=0).stop 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 shutdown_connections 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f48d40682f0 0x7f48d406a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48e0004d90 0x7f48f40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f415e3c0 0x7f48f415e830 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 --2- 172.21.15.53:0/1451314728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f415f290 0x7f48f4149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 >> 172.21.15.53:0/1451314728 conn(0x7f48f4072600 msgr2=0x7f48f4157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 shutdown_connections 2023-11-28T11:20:54.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:54.457+0000 7f48f93ff700 1 -- 172.21.15.53:0/1451314728 wait complete. 2023-11-28T11:20:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 32 2023-11-28T11:20:54.880 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:20:48.508614+0000', 'modified': '2023-11-28T11:20:51.523382+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55331': {'gid': 55331, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6834/1970232100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 1970232100}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 1970232100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:20:54.880 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:20:54.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:20:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.087+0000 7fc35f1ff700 1 Processor -- start 2023-11-28T11:20:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.087+0000 7fc35f1ff700 1 -- start start 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35f1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc35815c9b0 0x7fc3580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35f1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3580d1950 con 0x7fc358161340 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35f1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580b5800 con 0x7fc35815c9b0 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580ce140 con 0x7fc3580cd7c0 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc357fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc358161340 0x7fc3580d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36312/0 (socket says 172.21.15.53:36312) 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35cf9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc35815c9b0 0x7fc3580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc357fff700 1 -- 172.21.15.53:0/2301222315 learned_addr learned my addr 172.21.15.53:0/2301222315 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.088+0000 7fc35d79c700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35cf9b700 1 -- 172.21.15.53:0/2301222315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 msgr2=0x7fc3580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3580cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35cf9b700 1 -- 172.21.15.53:0/2301222315 >> v1:172.21.15.53:6789/0 conn(0x7fc358161340 legacy=0x7fc3580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35cf9b700 1 -- 172.21.15.53:0/2301222315 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580d24f0 con 0x7fc35815c9b0 2023-11-28T11:20:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35d79c700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc3577fe700 1 -- 172.21.15.53:0/2301222315 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446518972 0 0) 0x7fc3580d1950 con 0x7fc358161340 2023-11-28T11:20:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc35815c9b0 0x7fc3580b4bf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009840 tx=0x7fc34c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e428d2d8d7d230e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc3577fe700 1 -- 172.21.15.53:0/2301222315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c011830 con 0x7fc35815c9b0 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc3577fe700 1 -- 172.21.15.53:0/2301222315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c011990 con 0x7fc35815c9b0 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.089+0000 7fc3577fe700 1 -- 172.21.15.53:0/2301222315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c011d20 con 0x7fc35815c9b0 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 -- 172.21.15.53:0/2301222315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc35815c9b0 msgr2=0x7fc3580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc35815c9b0 0x7fc3580b4bf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009840 tx=0x7fc34c009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 -- 172.21.15.53:0/2301222315 shutdown_connections 2023-11-28T11:20:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/2301222315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc35815c9b0 0x7fc3580b4bf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 -- 172.21.15.53:0/2301222315 >> 172.21.15.53:0/2301222315 conn(0x7fc358072600 msgr2=0x7fc3580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 -- 172.21.15.53:0/2301222315 shutdown_connections 2023-11-28T11:20:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.090+0000 7fc35f1ff700 1 -- 172.21.15.53:0/2301222315 wait complete. 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.092+0000 7fc35f1ff700 1 Processor -- start 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.092+0000 7fc35f1ff700 1 -- start start 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3581579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35815c9b0 0x7fc35815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581f2f60 con 0x7fc358161340 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581f3240 con 0x7fc35815c9b0 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581f3630 con 0x7fc3580cd7c0 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38244/0 (socket says 172.21.15.53:38244) 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 -- 172.21.15.53:0/227878710 learned_addr learned my addr 172.21.15.53:0/227878710 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc357fff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35815c9b0 0x7fc35815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3581579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 -- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 msgr2=0x7fc3581579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3581579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 -- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35815c9b0 msgr2=0x7fc35815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35815c9b0 0x7fc35815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 -- 172.21.15.53:0/227878710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0094a0 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35d79c700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc350007e70 tx=0x7fc35000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc3557fa700 1 -- 172.21.15.53:0/227878710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000dd20 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc3557fa700 1 -- 172.21.15.53:0/227878710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35000f040 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3580d1300 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc3557fa700 1 -- 172.21.15.53:0/227878710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000dea0 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.094+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3580b5520 con 0x7fc358161340 2023-11-28T11:20:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.096+0000 7fc3557fa700 1 -- 172.21.15.53:0/227878710 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc35001c030 con 0x7fc358161340 2023-11-28T11:20:55.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc3557fa700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 0x7fc33c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc3557fa700 1 -- 172.21.15.53:0/227878710 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc350085c20 con 0x7fc358161340 2023-11-28T11:20:55.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 0x7fc33c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 0x7fc33c06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0090b0 tx=0x7fc34c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc372a88b80 1 Processor -- start 2023-11-28T11:20:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.099+0000 7fc372a88b80 1 -- start start 2023-11-28T11:20:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320dd3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320c77bc0 con 0x55d320db5130 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320ce6350 con 0x55d320db7810 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d320cdf880 con 0x55d320db8120 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.100+0000 7fc357fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38252/0 (socket says 172.21.15.53:38252) 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 -- 172.21.15.53:0/1012001693 learned_addr learned my addr 172.21.15.53:0/1012001693 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc35d79c700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320dd3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 -- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 msgr2=0x55d320dd3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320dd3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 -- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 msgr2=0x55d320d8da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 -- 172.21.15.53:0/1012001693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d320b615e0 con 0x55d320db5130 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc35d79c700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320dd3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc357fff700 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc62044b1c8bb558 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc3427fc700 1 -- 172.21.15.53:0/1012001693 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344001640 con 0x55d320db5130 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc3427fc700 1 -- 172.21.15.53:0/1012001693 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3440017a0 con 0x55d320db5130 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.101+0000 7fc3427fc700 1 -- 172.21.15.53:0/1012001693 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344001b50 con 0x55d320db5130 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 -- 172.21.15.53:0/1012001693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 msgr2=0x55d320db7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344007340 comp rx=0 tx=0).stop 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 -- 172.21.15.53:0/1012001693 shutdown_connections 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320dd3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1012001693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320db7be0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 -- 172.21.15.53:0/1012001693 >> 172.21.15.53:0/1012001693 conn(0x55d320d8ad60 msgr2=0x55d320d8b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 -- 172.21.15.53:0/1012001693 shutdown_connections 2023-11-28T11:20:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.102+0000 7fc372a88b80 1 -- 172.21.15.53:0/1012001693 wait complete. 2023-11-28T11:20:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.103+0000 7fc372a88b80 1 -- start start 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.103+0000 7fc372a88b80 1 Processor -- start 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320dbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320b638d0 con 0x55d320db5130 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320dd76c0 con 0x55d320db7810 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d320dd7230 con 0x55d320db8120 2023-11-28T11:20:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.104+0000 7fc372a88b80 10 client.0 _ll_register_callbacks cb 0x55d320b85240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38258/0 (socket says 172.21.15.53:38258) 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3174657106 learned_addr learned my addr 172.21.15.53:0/3174657106 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc357fff700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320dbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35d79c700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 msgr2=0x55d320db9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 msgr2=0x55d320dbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320dbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3174657106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344004b00 con 0x55d320db5130 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35d79c700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc3580675d0 tx=0x7fc34c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2924f3044d09f8a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc3377fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.105+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01a420 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc34c01f070 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d320cad140 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc34c01a580 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 10 client.6120 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d320b0dd90 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d320cad140 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc34c01a6e0 con 0x55d320db5130 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 10 client.6120 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 10 client.6120 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 20 client.6120 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:55.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320daedd0 0x55d320daf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 4 client.6120 mds_command: new command op to 55331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.106+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d320b63f10 con 0x55d320daedd0 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.107+0000 7fc357fff700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320daedd0 0x55d320daf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.107+0000 7fc357fff700 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320daedd0 0x55d320daf1a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51694e30eb750745 server_cookie=17fd560812415092 in_seq=0 out_seq=0 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.107+0000 7fc3377fe700 10 client.6120 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc3377fe700 1 -- 172.21.15.53:0/3174657106 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d320b63f10 con 0x55d320daedd0 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc3377fe700 10 client.6120 handle_command_reply: tid=0 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 client.6120 shutdown 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 msgr2=0x55d320dd6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc3580675d0 tx=0x7fc34c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 shutdown_connections 2023-11-28T11:20:55.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320daedd0 0x55d320daf1a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd6cf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc3377fe700 0 client.6120 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3174657106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320dbac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 >> 172.21.15.53:0/3174657106 conn(0x55d320d8ad60 msgr2=0x55d320d8c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 shutdown_connections 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.108+0000 7fc372a88b80 1 -- 172.21.15.53:0/3174657106 wait complete. 2023-11-28T11:20:55.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.109+0000 7fc372a88b80 20 client.6120 trim_cache size 0 max 16384 2023-11-28T11:20:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.128+0000 7fc372a88b80 1 Processor -- start 2023-11-28T11:20:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.128+0000 7fc372a88b80 1 -- start start 2023-11-28T11:20:55.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320dba330 con 0x55d320db5130 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320dafb20 con 0x55d320db7810 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d320dd9d60 con 0x55d320db8120 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38266/0 (socket says 172.21.15.53:38266) 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3039678202 learned_addr learned my addr 172.21.15.53:0/3039678202 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 msgr2=0x55d320db6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 msgr2=0x55d320d8b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 -- 172.21.15.53:0/3039678202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d320ca6140 con 0x55d320db5130 2023-11-28T11:20:55.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006cb0 tx=0x7fc34c00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9bbe8bd7880da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc342ffd700 1 -- 172.21.15.53:0/3039678202 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c003980 con 0x55d320db5130 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc342ffd700 1 -- 172.21.15.53:0/3039678202 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c02b040 con 0x55d320db5130 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.129+0000 7fc342ffd700 1 -- 172.21.15.53:0/3039678202 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c011630 con 0x55d320db5130 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 -- 172.21.15.53:0/3039678202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 msgr2=0x55d320dd5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006cb0 tx=0x7fc34c00b050 comp rx=0 tx=0).stop 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 -- 172.21.15.53:0/3039678202 shutdown_connections 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320db6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320dd5080 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 --2- 172.21.15.53:0/3039678202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d8b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 -- 172.21.15.53:0/3039678202 >> 172.21.15.53:0/3039678202 conn(0x55d320d8ad60 msgr2=0x55d320d871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 -- 172.21.15.53:0/3039678202 shutdown_connections 2023-11-28T11:20:55.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.130+0000 7fc372a88b80 1 -- 172.21.15.53:0/3039678202 wait complete. 2023-11-28T11:20:55.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.131+0000 7fc372a88b80 1 -- start start 2023-11-28T11:20:55.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.131+0000 7fc372a88b80 1 Processor -- start 2023-11-28T11:20:55.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320d87710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320dec410 con 0x55d320db5130 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d320e3aea0 con 0x55d320db7810 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d320e37540 con 0x55d320db8120 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc372a88b80 10 client.0 _ll_register_callbacks cb 0x55d320b85240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38278/0 (socket says 172.21.15.53:38278) 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 -- 172.21.15.53:0/1510170549 learned_addr learned my addr 172.21.15.53:0/1510170549 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35d79c700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320d87710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc357fff700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 -- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 msgr2=0x55d320d87710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320d87710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 -- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 msgr2=0x55d320d871d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d871d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.132+0000 7fc35cf9b700 1 -- 172.21.15.53:0/1510170549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d320b0dd90 con 0x55d320db5130 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc35cf9b700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc34c032040 tx=0x7fc34c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0bf6da86c87d36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc3377fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c003980 con 0x55d320db5130 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc34c01f040 con 0x55d320db5130 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d320c9d160 con 0x55d320db5130 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc34c011630 con 0x55d320db5130 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc372a88b80 10 client.6126 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d320b34200 con 0x55d320db5130 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.133+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d320c9d160 con 0x55d320db5130 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc34c01a420 con 0x55d320db5130 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 10 client.6126 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 10 client.6126 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 20 client.6126 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320dbaa10 0x55d320dbade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 4 client.6126 mds_command: new command op to 55331 tid=0[{"prefix": "status"}] 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d320c87170 con 0x55d320dbaa10 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc357fff700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320dbaa10 0x55d320dbade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc357fff700 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320dbaa10 0x55d320dbade0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e1eb072693cf33 server_cookie=cdbf904a80393892 in_seq=0 out_seq=0 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.134+0000 7fc3377fe700 10 client.6126 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc3377fe700 1 -- 172.21.15.53:0/1510170549 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d320c87170 con 0x55d320dbaa10 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc3377fe700 10 client.6126 handle_command_reply: tid=0 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 client.6126 shutdown 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 msgr2=0x55d320e37000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc34c032040 tx=0x7fc34c011000 comp rx=0 tx=0).stop 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 shutdown_connections 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d320dbaa10 0x55d320dbade0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d320db8120 0x55d320d87710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d320db5130 0x55d320e37000 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc3377fe700 0 client.6126 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 --2- 172.21.15.53:0/1510170549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d320db7810 0x55d320d871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 >> 172.21.15.53:0/1510170549 conn(0x55d320d8ad60 msgr2=0x55d320db6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 shutdown_connections 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 1 -- 172.21.15.53:0/1510170549 wait complete. 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.135+0000 7fc372a88b80 20 client.6126 trim_cache size 0 max 16384 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "38901cf1-eb03-4cf6-8c7d-44ad237eb202", 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 55331, 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-28T11:20:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.6243590909999996, 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 32, 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 61, 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 61, 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.616858708000001 2023-11-28T11:20:55.143 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 msgr2=0x7fc33c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 0x7fc33c06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0090b0 tx=0x7fc34c01c040 comp rx=0 tx=0).stop 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 msgr2=0x7fc35815e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc350007e70 tx=0x7fc35000d3b0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 shutdown_connections 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc33c0682f0 0x7fc33c06a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3580cd7c0 0x7fc3581579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc358161340 0x7fc35815e9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 --2- 172.21.15.53:0/227878710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35815c9b0 0x7fc35815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 >> 172.21.15.53:0/227878710 conn(0x7fc358072600 msgr2=0x7fc3580b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 shutdown_connections 2023-11-28T11:20:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.136+0000 7fc35f1ff700 1 -- 172.21.15.53:0/227878710 wait complete. 2023-11-28T11:20:55.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:20:55.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.356+0000 7f61a51ff700 1 Processor -- start 2023-11-28T11:20:55.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.356+0000 7f61a51ff700 1 -- start start 2023-11-28T11:20:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61a0161340 0x7f61a00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61a00d2320 0x7f61a00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00b5800 con 0x7f61a00d2320 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00d1950 con 0x7f61a0161340 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f61a51ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00b5520 con 0x7f61a015c9b0 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61a00d2320 0x7f61a00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61a00d2320 0x7f61a00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38286/0 (socket says 172.21.15.53:38286) 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 -- 172.21.15.53:0/4145588248 learned_addr learned my addr 172.21.15.53:0/4145588248 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f6197fff700 1 --2- 172.21.15.53:0/4145588248 >> v2:172.21.15.157:3300/0 conn(0x7f61a0161340 0x7f61a00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619ed9d700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 -- 172.21.15.53:0/4145588248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 msgr2=0x7f61a00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 -- 172.21.15.53:0/4145588248 >> v2:172.21.15.157:3300/0 conn(0x7f61a0161340 msgr2=0x7f61a00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 --2- 172.21.15.53:0/4145588248 >> v2:172.21.15.157:3300/0 conn(0x7f61a0161340 0x7f61a00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 -- 172.21.15.53:0/4145588248 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00d1300 con 0x7f61a00d2320 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619ed9d700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.358+0000 7f619f59e700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00cd7c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f61a006bc00 tx=0x7f619800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d225767fead3e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f619e59c700 1 -- 172.21.15.53:0/4145588248 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800e070 con 0x7f61a00d2320 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f619e59c700 1 -- 172.21.15.53:0/4145588248 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800b9c0 con 0x7f61a00d2320 2023-11-28T11:20:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f619e59c700 1 -- 172.21.15.53:0/4145588248 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800bd70 con 0x7f61a00d2320 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 -- 172.21.15.53:0/4145588248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 msgr2=0x7f61a00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00cd7c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f61a006bc00 tx=0x7f619800b330 comp rx=0 tx=0).stop 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 -- 172.21.15.53:0/4145588248 shutdown_connections 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 --2- 172.21.15.53:0/4145588248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00cd7c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 --2- 172.21.15.53:0/4145588248 >> v2:172.21.15.157:3300/0 conn(0x7f61a0161340 0x7f61a00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 -- 172.21.15.53:0/4145588248 >> 172.21.15.53:0/4145588248 conn(0x7f61a0072600 msgr2=0x7f61a00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 -- 172.21.15.53:0/4145588248 shutdown_connections 2023-11-28T11:20:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.359+0000 7f61a51ff700 1 -- 172.21.15.53:0/4145588248 wait complete. 2023-11-28T11:20:55.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.361+0000 7f61a51ff700 1 Processor -- start 2023-11-28T11:20:55.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.361+0000 7f61a51ff700 1 -- start start 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.362+0000 7f61a51ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f61a51ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f61a51ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61a0161340 0x7f61a0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f61a51ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01536d0 con 0x7f61a00d2320 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f61a51ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152ac0 con 0x7f61a0161340 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f61a51ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152da0 con 0x7f61a015c9b0 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38292/0 (socket says 172.21.15.53:38292) 2023-11-28T11:20:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 -- 172.21.15.53:0/3331106508 learned_addr learned my addr 172.21.15.53:0/3331106508 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f6197fff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619f59e700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61a0161340 0x7f61a0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 -- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 msgr2=0x7f61a01506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a01506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 -- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61a0161340 msgr2=0x7f61a0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61a0161340 0x7f61a0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.363+0000 7f619ed9d700 1 -- 172.21.15.53:0/3331106508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198009b30 con 0x7f61a00d2320 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f619ed9d700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f619000c7f0 tx=0x7f619000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011800 con 0x7f61a00d2320 2023-11-28T11:20:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6190011960 con 0x7f61a00d2320 2023-11-28T11:20:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011cf0 con 0x7f61a00d2320 2023-11-28T11:20:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a00b5130 con 0x7f61a00d2320 2023-11-28T11:20:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.364+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a00d15e0 con 0x7f61a00d2320 2023-11-28T11:20:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.365+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a00b52c0 con 0x7f61a00d2320 2023-11-28T11:20:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.366+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6190021070 con 0x7f61a00d2320 2023-11-28T11:20:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.367+0000 7f61977fe700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 0x7f618006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.367+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6190085a50 con 0x7f61a00d2320 2023-11-28T11:20:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.367+0000 7f6197fff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 0x7f618006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.368+0000 7f6197fff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 0x7f618006a7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61a00c96b0 tx=0x7f6188008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.372+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61900521c0 con 0x7f61a00d2320 2023-11-28T11:20:55.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.623+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61a00745c0 con 0x7f61a00d2320 2023-11-28T11:20:55.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.623+0000 7f61977fe700 1 -- 172.21.15.53:0/3331106508 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f619001f060 con 0x7f61a00d2320 2023-11-28T11:20:55.625 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:20:55.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 msgr2=0x7f618006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 0x7f618006a7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61a00c96b0 tx=0x7f6188008040 comp rx=0 tx=0).stop 2023-11-28T11:20:55.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 msgr2=0x7f61a00c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f619000c7f0 tx=0x7f619000cb00 comp rx=0 tx=0).stop 2023-11-28T11:20:55.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 shutdown_connections 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6180068320 0x7f618006a7d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61a0161340 0x7f61a0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a00d2320 0x7f61a00c6ca0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 --2- 172.21.15.53:0/3331106508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a015c9b0 0x7f61a01506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 >> 172.21.15.53:0/3331106508 conn(0x7f61a0072600 msgr2=0x7f61a015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 shutdown_connections 2023-11-28T11:20:55.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:55.626+0000 7f61a51ff700 1 -- 172.21.15.53:0/3331106508 wait complete. 2023-11-28T11:20:55.646 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:20:55.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:20:55.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-28T11:20:55.646 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:20:55.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:20:55.647 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:20:55.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:20:55.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:20:55.647 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:20:55.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:55.647 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-28T11:20:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41963sec preferred_lft 41963sec 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:20:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::892:d6ff:fe9c:c696/64 scope link 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:6e:80:63:4c:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::86e:80ff:fe63:4cd1/64 scope link 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:e4:5d:14:b9:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:20:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::68e4:5dff:fe14:b995/64 scope link 2023-11-28T11:20:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:20:55.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:55.695 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:20:55.798 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-28T11:20:55.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:55.802 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:20:55.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:20:55.850 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-28T11:20:56.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.025 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:20:56.069 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:20:56.070 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:20:56.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:20:56.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-11-28T11:20:56.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:20:56.180 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:20:56.223 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:20:56.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:20:56.223 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:20:56.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:20:56.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:20:56.453 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:20:56.453 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:20:56.453 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:20:56.472 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:20:56.472 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:20:56.473 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:20:56.473 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:20:56.473 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:20:56.494 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:20:56.494 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:20:56.494 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2023-11-28T11:20:56.494 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-11-28T11:20:56.494 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:20:56.495 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790" 2023-11-28T11:20:56.495 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.157:6789,172.21.15.157:6790:/ 2023-11-28T11:20:56.495 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=38901cf1-eb03-4cf6-8c7d-44ad237eb202" will pass to kernel 2023-11-28T11:20:56.500 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:20:56.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:20:56.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.573 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:20:56.670 INFO:teuthology.orchestra.run.smithi053.stdout:client6135 2023-11-28T11:20:56.671 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:20:56.671 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:20:56.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.716 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:20:56.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:20:56.805 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:20:56.851 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:20:56.851 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-11-28T11:20:56.852 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2023-11-28T11:20:56.852 DEBUG:teuthology.orchestra.run.smithi053:> 2023-11-28T11:20:56.852 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:20:56.852 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-11-28T11:20:57.005 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2023-11-28T11:20:57.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:57.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.429+0000 7f39d67ff700 1 Processor -- start 2023-11-28T11:20:57.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.430+0000 7f39d67ff700 1 -- start start 2023-11-28T11:20:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d67ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39d0161340 0x7f39d00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d67ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 0x7f39d00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d67ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d00d1950 con 0x7f39d015c9b0 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d67ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00b5800 con 0x7f39d0161340 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d67ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00b5300 con 0x7f39d00d2320 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39d0161340 0x7f39d00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39d0161340 0x7f39d00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38306/0 (socket says 172.21.15.53:38306) 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 -- 172.21.15.53:0/2768450117 learned_addr learned my addr 172.21.15.53:0/2768450117 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d4d9c700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 0x7f39d00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 -- 172.21.15.53:0/2768450117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 msgr2=0x7f39d00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 0x7f39d00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 -- 172.21.15.53:0/2768450117 >> v1:172.21.15.157:6789/0 conn(0x7f39d015c9b0 legacy=0x7f39d00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39cf7fe700 1 -- 172.21.15.53:0/2768450117 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00b5520 con 0x7f39d0161340 2023-11-28T11:20:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.431+0000 7f39d4d9c700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 0x7f39d00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39cf7fe700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d0161340 0x7f39d00d1de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098f0 tx=0x7f39c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d629020ee43cbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39ceffd700 1 -- 172.21.15.53:0/2768450117 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4004230 con 0x7f39d0161340 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39ceffd700 1 -- 172.21.15.53:0/2768450117 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4004390 con 0x7f39d0161340 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39ceffd700 1 -- 172.21.15.53:0/2768450117 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40036a0 con 0x7f39d0161340 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39d67ff700 1 -- 172.21.15.53:0/2768450117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d0161340 msgr2=0x7f39d00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.432+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d0161340 0x7f39d00d1de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098f0 tx=0x7f39c4009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 -- 172.21.15.53:0/2768450117 shutdown_connections 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d00d2320 0x7f39d00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2768450117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d0161340 0x7f39d00d1de0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 -- 172.21.15.53:0/2768450117 >> 172.21.15.53:0/2768450117 conn(0x7f39d0072600 msgr2=0x7f39d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 -- 172.21.15.53:0/2768450117 shutdown_connections 2023-11-28T11:20:57.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.433+0000 7f39d67ff700 1 -- 172.21.15.53:0/2768450117 wait complete. 2023-11-28T11:20:57.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.434+0000 7f39d67ff700 1 Processor -- start 2023-11-28T11:20:57.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.435+0000 7f39d67ff700 1 -- start start 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.435+0000 7f39d67ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d67ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d015c9b0 0x7f39d0147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d67ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 0x7f39d01f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d67ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d0144080 con 0x7f39d00d2320 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d67ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01442d0 con 0x7f39d015c9b0 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d67ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01445b0 con 0x7f39d0161340 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38312/0 (socket says 172.21.15.53:38312) 2023-11-28T11:20:57.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 -- 172.21.15.53:0/2797274843 learned_addr learned my addr 172.21.15.53:0/2797274843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cf7fe700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d015c9b0 0x7f39d0147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d4d9c700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 0x7f39d01f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 -- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 msgr2=0x7f39d01f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 0x7f39d01f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 -- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d015c9b0 msgr2=0x7f39d0147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d015c9b0 0x7f39d0147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39cffff700 1 -- 172.21.15.53:0/2797274843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009550 con 0x7f39d00d2320 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.436+0000 7f39d4d9c700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 0x7f39d01f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39cffff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b480 tx=0x7f39c000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0011830 con 0x7f39d00d2320 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c00119b0 con 0x7f39d00d2320 2023-11-28T11:20:57.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0011d60 con 0x7f39d00d2320 2023-11-28T11:20:57.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d00d15e0 con 0x7f39d00d2320 2023-11-28T11:20:57.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.437+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d00d1300 con 0x7f39d00d2320 2023-11-28T11:20:57.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.438+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d015cd80 con 0x7f39d00d2320 2023-11-28T11:20:57.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.439+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39c0021070 con 0x7f39d00d2320 2023-11-28T11:20:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.440+0000 7f39ccff9700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.440+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39c0085c30 con 0x7f39d00d2320 2023-11-28T11:20:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.440+0000 7f39cf7fe700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.441+0000 7f39cf7fe700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 0x7f39bc06a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009130 tx=0x7f39c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.444+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39c00523d0 con 0x7f39d00d2320 2023-11-28T11:20:57.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.697+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39d00745c0 con 0x7f39d00d2320 2023-11-28T11:20:57.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.698+0000 7f39ccff9700 1 -- 172.21.15.53:0/2797274843 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f39c0015070 con 0x7f39d00d2320 2023-11-28T11:20:57.699 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:57.699 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:20:51.523382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:20:57.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 msgr2=0x7f39bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 0x7f39bc06a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009130 tx=0x7f39c4008030 comp rx=0 tx=0).stop 2023-11-28T11:20:57.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 msgr2=0x7f39d00c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b480 tx=0x7f39c000b790 comp rx=0 tx=0).stop 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 shutdown_connections 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d0161340 0x7f39d01f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00d2320 0x7f39d00c86e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 --2- 172.21.15.53:0/2797274843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d015c9b0 0x7f39d0147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 >> 172.21.15.53:0/2797274843 conn(0x7f39d0072600 msgr2=0x7f39d00b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 shutdown_connections 2023-11-28T11:20:57.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:57.700+0000 7f39d67ff700 1 -- 172.21.15.53:0/2797274843 wait complete. 2023-11-28T11:20:57.703 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 32 2023-11-28T11:20:57.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:20:57.953 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2023-11-28T11:20:57.954 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:20:57.954 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:20:58.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.040+0000 7fe2a2f7d080 1 Processor -- start 2023-11-28T11:20:58.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.040+0000 7fe2a2f7d080 1 -- start start 2023-11-28T11:20:58.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.041+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be14424a60 0x55be14452bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.041+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.041+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be14422940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1435efe0 con 0x55be144278c0 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be144fade0 con 0x55be14424a60 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be1444eaa0 con 0x55be14454fe0 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38334/0 (socket says 172.21.15.53:38334) 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 -- 172.21.15.53:0/1322348966 learned_addr learned my addr 172.21.15.53:0/1322348966 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe294d9c700 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be14422940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe29459b700 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be14424a60 0x55be14452bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 -- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 msgr2=0x55be14422940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be14422940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 -- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be14424a60 msgr2=0x55be14452bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be14424a60 0x55be14452bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.042+0000 7fe293d9a700 1 -- 172.21.15.53:0/1322348966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288005de0 con 0x55be144278c0 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe293d9a700 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe288015040 tx=0x7fe28800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f458a92880346bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe292d98700 1 -- 172.21.15.53:0/1322348966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288008670 con 0x55be144278c0 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe292d98700 1 -- 172.21.15.53:0/1322348966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288008b40 con 0x55be144278c0 2023-11-28T11:20:58.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe292d98700 1 -- 172.21.15.53:0/1322348966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288005580 con 0x55be144278c0 2023-11-28T11:20:58.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/1322348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 msgr2=0x55be14422400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe288015040 tx=0x7fe28800d000 comp rx=0 tx=0).stop 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/1322348966 shutdown_connections 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be14422940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be14424a60 0x55be14452bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/1322348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be144278c0 0x55be14422400 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/1322348966 >> 172.21.15.53:0/1322348966 conn(0x55be144513e0 msgr2=0x55be144539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/1322348966 shutdown_connections 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.043+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/1322348966 wait complete. 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.044+0000 7fe2a2f7d080 1 Processor -- start 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.044+0000 7fe2a2f7d080 1 -- start start 2023-11-28T11:20:58.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 0x55be14592ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be144278c0 0x55be144efea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be144517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1446b950 con 0x55be14424a60 2023-11-28T11:20:58.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be144539b0 con 0x55be144278c0 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.045+0000 7fe2a2f7d080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be144536d0 con 0x55be14454fe0 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 0x55be14592ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 0x55be14592ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38342/0 (socket says 172.21.15.53:38342) 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 -- 172.21.15.53:0/3825286474 learned_addr learned my addr 172.21.15.53:0/3825286474 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe294d9c700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be144517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe293d9a700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be144278c0 0x55be144efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 -- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 msgr2=0x55be144517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be14454fe0 0x55be144517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 -- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be144278c0 msgr2=0x55be144efea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be144278c0 0x55be144efea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 -- 172.21.15.53:0/3825286474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be143914f0 con 0x55be14424a60 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe293d9a700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be144278c0 0x55be144efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:58.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe29459b700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 0x55be14592ff0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e70 tx=0x7fe2840086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe291d96700 1 -- 172.21.15.53:0/3825286474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840145f0 con 0x55be14424a60 2023-11-28T11:20:58.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe291d96700 1 -- 172.21.15.53:0/3825286474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe284010040 con 0x55be14424a60 2023-11-28T11:20:58.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.046+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be14377120 con 0x55be14424a60 2023-11-28T11:20:58.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.047+0000 7fe291d96700 1 -- 172.21.15.53:0/3825286474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284014990 con 0x55be14424a60 2023-11-28T11:20:58.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.047+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be1436a830 con 0x55be14424a60 2023-11-28T11:20:58.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.048+0000 7fe291d96700 1 -- 172.21.15.53:0/3825286474 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe28401b030 con 0x55be14424a60 2023-11-28T11:20:58.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.049+0000 7fe291d96700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe280066c10 0x7fe2800690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.049+0000 7fe291d96700 1 -- 172.21.15.53:0/3825286474 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe284083f90 con 0x55be14424a60 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.050+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x55be1446e530 0x55be14584270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.050+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61) v8 -- 0x55be144f91a0 con 0x55be1446e530 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.050+0000 7fe293d9a700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe280066c10 0x7fe2800690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.050+0000 7fe294d9c700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x55be1446e530 0x55be14584270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.050+0000 7fe293d9a700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe280066c10 0x7fe2800690c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe288020000 tx=0x7fe288010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe294d9c700 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x55be1446e530 0x55be14584270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe294d9c700 1 -- 172.21.15.53:0/3825286474 <== osd.6 v2:172.21.15.157:6815/132919 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7fe28c001540 con 0x55be1446e530 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x55be1446e530 msgr2=0x55be14584270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x55be1446e530 0x55be14584270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe280066c10 msgr2=0x7fe2800690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe280066c10 0x7fe2800690c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe288020000 tx=0x7fe288010000 comp rx=0 tx=0).stop 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 msgr2=0x55be14592ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 --2- 172.21.15.53:0/3825286474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be14424a60 0x55be14592ff0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e70 tx=0x7fe2840086e0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe294d9c700 1 -- 172.21.15.53:0/3825286474 reap_dead start 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 shutdown_connections 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.051+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 >> 172.21.15.53:0/3825286474 conn(0x55be144513e0 msgr2=0x55be14468930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.052+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 shutdown_connections 2023-11-28T11:20:58.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.052+0000 7fe2a2f7d080 1 -- 172.21.15.53:0/3825286474 wait complete. 2023-11-28T11:20:58.059 INFO:teuthology.orchestra.run.smithi053.stdout:Xfiledir_test 2023-11-28T11:20:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:58.060 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:20:58.168 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2023-11-28T11:20:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2023-11-28T11:20:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "file", 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "dir_test", 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-28T11:20:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 10, 2023-11-28T11:20:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2023-11-28T11:20:58.170 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:20:58.173 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:20:58.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 Processor -- start 2023-11-28T11:20:58.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 -- start start 2023-11-28T11:20:58.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e0943930 0x5636e09d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e093f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e0835fe0 con 0x5636e09d3260 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e09418c0 con 0x5636e09d3630 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.259+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e0926ef0 con 0x5636e0943930 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38352/0 (socket says 172.21.15.53:38352) 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 -- 172.21.15.53:0/4092870427 learned_addr learned my addr 172.21.15.53:0/4092870427 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 -- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e0943930 msgr2=0x5636e09d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e0943930 0x5636e09d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 -- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 msgr2=0x5636e093f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e093f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 -- 172.21.15.53:0/4092870427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b4005d20 con 0x5636e09d3260 2023-11-28T11:20:58.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009750 tx=0x7fa4b8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e666058fce59c872 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.260+0000 7fa4c2d98700 1 -- 172.21.15.53:0/4092870427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800f040 con 0x5636e09d3260 2023-11-28T11:20:58.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4c2d98700 1 -- 172.21.15.53:0/4092870427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b80088d0 con 0x5636e09d3260 2023-11-28T11:20:58.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4c2d98700 1 -- 172.21.15.53:0/4092870427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8005e10 con 0x5636e09d3260 2023-11-28T11:20:58.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/4092870427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 msgr2=0x5636e093ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009750 tx=0x7fa4b8005a00 comp rx=0 tx=0).stop 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/4092870427 shutdown_connections 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e0943930 0x5636e09d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e093f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/4092870427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e09d3260 0x5636e093ed40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/4092870427 >> 172.21.15.53:0/4092870427 conn(0x5636e09479e0 msgr2=0x5636e09d6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/4092870427 shutdown_connections 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.261+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/4092870427 wait complete. 2023-11-28T11:20:58.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.262+0000 7fa4d2eb9080 1 Processor -- start 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.262+0000 7fa4d2eb9080 1 -- start start 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 0x5636e09cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e09d3260 0x5636e09b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e09d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e09cdb20 con 0x5636e0943930 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e09b83d0 con 0x5636e09d3630 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4d2eb9080 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e09ccf10 con 0x5636e09d3260 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4c459b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 0x5636e09cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4c459b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 0x5636e09cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38360/0 (socket says 172.21.15.53:38360) 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.263+0000 7fa4c459b700 1 -- 172.21.15.53:0/3076883060 learned_addr learned my addr 172.21.15.53:0/3076883060 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e09d3260 0x5636e09b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c4d9c700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e09d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 -- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e09d3260 msgr2=0x5636e09b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e09d3260 0x5636e09b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 -- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 msgr2=0x5636e09d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e09d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 -- 172.21.15.53:0/3076883060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8009410 con 0x5636e0943930 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5636e09d3260 0x5636e09b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c4d9c700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5636e09d3630 0x5636e09d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c459b700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 0x5636e09cdfb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c1d96700 1 -- 172.21.15.53:0/3076883060 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b401e650 con 0x5636e0943930 2023-11-28T11:20:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c1d96700 1 -- 172.21.15.53:0/3076883060 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b401e7f0 con 0x5636e0943930 2023-11-28T11:20:58.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636e084e120 con 0x5636e0943930 2023-11-28T11:20:58.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4c1d96700 1 -- 172.21.15.53:0/3076883060 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b401eba0 con 0x5636e0943930 2023-11-28T11:20:58.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.264+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e0841830 con 0x5636e0943930 2023-11-28T11:20:58.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.266+0000 7fa4c1d96700 1 -- 172.21.15.53:0/3076883060 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4b401ed00 con 0x5636e0943930 2023-11-28T11:20:58.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.267+0000 7fa4c1d96700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4b0066c10 0x7fa4b00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.267+0000 7fa4c1d96700 1 -- 172.21.15.53:0/3076883060 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4b4086480 con 0x5636e0943930 2023-11-28T11:20:58.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.267+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x5636e09d4280 0x5636e09cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.267+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4b0066c10 0x7fa4b00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.267+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61) v8 -- 0x5636e09d0030 con 0x5636e09d4280 2023-11-28T11:20:58.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.268+0000 7fa4c4d9c700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x5636e09d4280 0x5636e09cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.268+0000 7fa4c3d9a700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4b0066c10 0x7fa4b00690c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5636e0941fa0 tx=0x7fa4b8012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.268+0000 7fa4c4d9c700 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x5636e09d4280 0x5636e09cfaf0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4c4d9c700 1 -- 172.21.15.53:0/3076883060 <== osd.6 v2:172.21.15.157:6815/132919 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v61'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa4bc001790 con 0x5636e09d4280 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x5636e09d4280 msgr2=0x5636e09cfaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x5636e09d4280 0x5636e09cfaf0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4b0066c10 msgr2=0x7fa4b00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4b0066c10 0x7fa4b00690c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5636e0941fa0 tx=0x7fa4b8012000 comp rx=0 tx=0).stop 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 msgr2=0x5636e09cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 --2- 172.21.15.53:0/3076883060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636e0943930 0x5636e09cdfb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4008f40 comp rx=0 tx=0).stop 2023-11-28T11:20:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4c4d9c700 1 -- 172.21.15.53:0/3076883060 reap_dead start 2023-11-28T11:20:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 shutdown_connections 2023-11-28T11:20:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 >> 172.21.15.53:0/3076883060 conn(0x5636e09479e0 msgr2=0x5636e0947db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 shutdown_connections 2023-11-28T11:20:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.270+0000 7fa4d2eb9080 1 -- 172.21.15.53:0/3076883060 wait complete. 2023-11-28T11:20:58.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:20:58.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.480+0000 7fcee95ff700 1 Processor -- start 2023-11-28T11:20:58.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.480+0000 7fcee95ff700 1 -- start start 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 0x7fcee4161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee95ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcee40cded0 0x7fcee40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee95ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee415e0a0 con 0x7fcee415dc30 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee95ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee415ab30 con 0x7fcee40cded0 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4160d70 con 0x7fcee415a760 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee37fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcee40cded0 0x7fcee40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee27fc700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcee415dc30 0x7fcee40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36322/0 (socket says 172.21.15.53:36322) 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee27fc700 1 -- 172.21.15.53:0/2362690940 learned_addr learned my addr 172.21.15.53:0/2362690940 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.481+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 0x7fcee4161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee37fe700 1 -- 172.21.15.53:0/2362690940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 msgr2=0x7fcee4161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee37fe700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 0x7fcee4161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee37fe700 1 -- 172.21.15.53:0/2362690940 >> v1:172.21.15.53:6789/0 conn(0x7fcee415dc30 legacy=0x7fcee40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee37fe700 1 -- 172.21.15.53:0/2362690940 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40b5130 con 0x7fcee40cded0 2023-11-28T11:20:58.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee1ffb700 1 -- 172.21.15.53:0/2362690940 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038694590 0 0) 0x7fcee415e0a0 con 0x7fcee415dc30 2023-11-28T11:20:58.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 0x7fcee4161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:58.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee37fe700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee40cded0 0x7fcee40b4bf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009d90 tx=0x7fcedc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31991a032e91c40a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee1ffb700 1 -- 172.21.15.53:0/2362690940 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00e070 con 0x7fcee40cded0 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee1ffb700 1 -- 172.21.15.53:0/2362690940 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc00bbd0 con 0x7fcee40cded0 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.482+0000 7fcee1ffb700 1 -- 172.21.15.53:0/2362690940 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc012430 con 0x7fcee40cded0 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 -- 172.21.15.53:0/2362690940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee40cded0 msgr2=0x7fcee40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee40cded0 0x7fcee40b4bf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009d90 tx=0x7fcedc00b330 comp rx=0 tx=0).stop 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 -- 172.21.15.53:0/2362690940 shutdown_connections 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee40cded0 0x7fcee40b4bf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 --2- 172.21.15.53:0/2362690940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee415a760 0x7fcee4161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 -- 172.21.15.53:0/2362690940 >> 172.21.15.53:0/2362690940 conn(0x7fcee4072620 msgr2=0x7fcee40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 -- 172.21.15.53:0/2362690940 shutdown_connections 2023-11-28T11:20:58.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.483+0000 7fcee95ff700 1 -- 172.21.15.53:0/2362690940 wait complete. 2023-11-28T11:20:58.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.485+0000 7fcee95ff700 1 Processor -- start 2023-11-28T11:20:58.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.486+0000 7fcee95ff700 1 -- start start 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee40cded0 0x7fcee41f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 0x7fcee415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee414cee0 con 0x7fcee415dc30 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41f23c0 con 0x7fcee415a760 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee95ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee415bdf0 con 0x7fcee40cded0 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38382/0 (socket says 172.21.15.53:38382) 2023-11-28T11:20:58.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee37fe700 1 -- 172.21.15.53:0/3075956777 learned_addr learned my addr 172.21.15.53:0/3075956777 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee27fc700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 0x7fcee415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.487+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee40cded0 0x7fcee41f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 -- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee40cded0 msgr2=0x7fcee41f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee40cded0 0x7fcee41f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 -- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 msgr2=0x7fcee415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 0x7fcee415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 -- 172.21.15.53:0/3075956777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc009a70 con 0x7fcee415dc30 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee27fc700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 0x7fcee415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee37fe700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcedc014040 tx=0x7fcedc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcecb7fe700 1 -- 172.21.15.53:0/3075956777 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc012430 con 0x7fcee415dc30 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcecb7fe700 1 -- 172.21.15.53:0/3075956777 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc020070 con 0x7fcee415dc30 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee40b5670 con 0x7fcee415dc30 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcecb7fe700 1 -- 172.21.15.53:0/3075956777 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc004280 con 0x7fcee415dc30 2023-11-28T11:20:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.488+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40b5390 con 0x7fcee415dc30 2023-11-28T11:20:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.490+0000 7fcecb7fe700 1 -- 172.21.15.53:0/3075956777 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcedc02e070 con 0x7fcee415dc30 2023-11-28T11:20:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.491+0000 7fcecb7fe700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 0x7fced006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.491+0000 7fcecb7fe700 1 -- 172.21.15.53:0/3075956777 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcedc022070 con 0x7fcee415dc30 2023-11-28T11:20:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.492+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 0x7fced006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.492+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 0x7fced006a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcee4153de0 tx=0x7fced4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.494+0000 7fcefce36b80 1 Processor -- start 2023-11-28T11:20:58.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.494+0000 7fcefce36b80 1 -- start start 2023-11-28T11:20:58.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9abead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a99a8c70 con 0x55d8a9ae91d0 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9a17400 con 0x55d8a9ae61e0 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9a10930 con 0x55d8a9ae88c0 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38392/0 (socket says 172.21.15.53:38392) 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcee37fe700 1 -- 172.21.15.53:0/199252469 learned_addr learned my addr 172.21.15.53:0/199252469 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9abead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.495+0000 7fcee27fc700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 -- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 msgr2=0x55d8a9abead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9abead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 -- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 msgr2=0x55d8a9ae8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 -- 172.21.15.53:0/199252469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a9892690 con 0x55d8a9ae91d0 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee27fc700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcee37fe700 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcedc05fb00 tx=0x7fcedc057bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255a37bff6b007c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcec8ff9700 1 -- 172.21.15.53:0/199252469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc06f1c0 con 0x55d8a9ae91d0 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcec8ff9700 1 -- 172.21.15.53:0/199252469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc05fba0 con 0x55d8a9ae91d0 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.496+0000 7fcec8ff9700 1 -- 172.21.15.53:0/199252469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc053d90 con 0x55d8a9ae91d0 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 -- 172.21.15.53:0/199252469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 msgr2=0x55d8a9b04820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcedc05fb00 tx=0x7fcedc057bb0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 -- 172.21.15.53:0/199252469 shutdown_connections 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9abead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9b04820 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 --2- 172.21.15.53:0/199252469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 -- 172.21.15.53:0/199252469 >> 172.21.15.53:0/199252469 conn(0x55d8a9abbe10 msgr2=0x55d8a9abc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 -- 172.21.15.53:0/199252469 shutdown_connections 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.497+0000 7fcefce36b80 1 -- 172.21.15.53:0/199252469 wait complete. 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.498+0000 7fcefce36b80 1 -- start start 2023-11-28T11:20:58.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.498+0000 7fcefce36b80 1 Processor -- start 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ad7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9894980 con 0x55d8a9ae91d0 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9adaf20 con 0x55d8a9ae61e0 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9afb970 con 0x55d8a9ae88c0 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcefce36b80 10 client.0 _ll_register_callbacks cb 0x55d8a98b62f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38400/0 (socket says 172.21.15.53:38400) 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcee37fe700 1 -- 172.21.15.53:0/1128911169 learned_addr learned my addr 172.21.15.53:0/1128911169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcee27fc700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ad7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.499+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 -- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 msgr2=0x55d8a9ad7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ad7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 -- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 msgr2=0x55d8a9ae7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 -- 172.21.15.53:0/1128911169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40b5390 con 0x55d8a9ae91d0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcee37fe700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcedc057ed0 tx=0x7fcedc05d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b4f9594a03856 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcec1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc06f280 con 0x55d8a9ae91d0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcedc056f00 con 0x55d8a9ae91d0 2023-11-28T11:20:58.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8a99de1f0 con 0x55d8a9ae91d0 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcedc0570c0 con 0x55d8a9ae91d0 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcefce36b80 10 client.6171 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d8a983ee40 con 0x55d8a9ae91d0 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.500+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc074020 con 0x55d8a9ae91d0 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcedc06b250 con 0x55d8a9ae91d0 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 10 client.6171 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 10 client.6171 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 20 client.6171 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9ada660 0x55d8a9adaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 4 client.6171 mds_command: new command op to 55331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8a9894fc0 con 0x55d8a9ada660 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9ada660 0x55d8a9adaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcec1ffb700 10 client.6171 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.501+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9ada660 0x55d8a9adaa30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6affee2a64dc4dc9 server_cookie=c558d60b0138db19 in_seq=0 out_seq=0 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcec1ffb700 1 -- 172.21.15.53:0/1128911169 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d8a9894fc0 con 0x55d8a9ada660 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcec1ffb700 10 client.6171 handle_command_reply: tid=0 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 client.6171 shutdown 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 msgr2=0x55d8a9ada120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcedc057ed0 tx=0x7fcedc05d000 comp rx=0 tx=0).stop 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 shutdown_connections 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9ada660 0x55d8a9adaa30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.502+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ada120 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcec1ffb700 0 client.6171 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9ae7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 1 --2- 172.21.15.53:0/1128911169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ad7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 >> 172.21.15.53:0/1128911169 conn(0x55d8a9abbe10 msgr2=0x55d8a9abd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 shutdown_connections 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 1 -- 172.21.15.53:0/1128911169 wait complete. 2023-11-28T11:20:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.503+0000 7fcefce36b80 20 client.6171 trim_cache size 0 max 16384 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.520+0000 7fcefce36b80 1 Processor -- start 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.520+0000 7fcefce36b80 1 -- start start 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9adb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae91d0 0x55d8a9ab81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9afca90 con 0x55d8a9ae88c0 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9adbd20 con 0x55d8a9ae61e0 2023-11-28T11:20:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9abd3c0 con 0x55d8a9ae91d0 2023-11-28T11:20:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcee27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcee27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38408/0 (socket says 172.21.15.53:38408) 2023-11-28T11:20:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.521+0000 7fcee27fc700 1 -- 172.21.15.53:0/103475807 learned_addr learned my addr 172.21.15.53:0/103475807 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 -- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae91d0 msgr2=0x55d8a9ab81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae91d0 0x55d8a9ab81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 -- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 msgr2=0x55d8a9adb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9adb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 -- 172.21.15.53:0/103475807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a99d71f0 con 0x55d8a9ae88c0 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcee27fc700 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fced8005da0 tx=0x7fced8007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bb9e96426445da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcec97fa700 1 -- 172.21.15.53:0/103475807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801a070 con 0x55d8a9ae88c0 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcec97fa700 1 -- 172.21.15.53:0/103475807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800d070 con 0x55d8a9ae88c0 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.522+0000 7fcec97fa700 1 -- 172.21.15.53:0/103475807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8004a80 con 0x55d8a9ae88c0 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 -- 172.21.15.53:0/103475807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 msgr2=0x55d8a9abe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fced8005da0 tx=0x7fced8007330 comp rx=0 tx=0).stop 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 -- 172.21.15.53:0/103475807 shutdown_connections 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae91d0 0x55d8a9ab81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae61e0 0x55d8a9adb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 --2- 172.21.15.53:0/103475807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae88c0 0x55d8a9abe8e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 -- 172.21.15.53:0/103475807 >> 172.21.15.53:0/103475807 conn(0x55d8a9abbe10 msgr2=0x55d8a9abd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 -- 172.21.15.53:0/103475807 shutdown_connections 2023-11-28T11:20:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.523+0000 7fcefce36b80 1 -- 172.21.15.53:0/103475807 wait complete. 2023-11-28T11:20:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.524+0000 7fcefce36b80 1 -- start start 2023-11-28T11:20:58.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.524+0000 7fcefce36b80 1 Processor -- start 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ae4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ae4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9afd230 con 0x55d8a9ae61e0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9abe390 con 0x55d8a9ae91d0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a9ad6c90 con 0x55d8a9ae88c0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcefce36b80 10 client.0 _ll_register_callbacks cb 0x55d8a98b62f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38424/0 (socket says 172.21.15.53:38424) 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 -- 172.21.15.53:0/3623187556 learned_addr learned my addr 172.21.15.53:0/3623187556 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee27fc700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ae4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee37fe700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ae4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 -- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 msgr2=0x55d8a9ae4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ae4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.525+0000 7fcee2ffd700 1 -- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 msgr2=0x55d8a9ae4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ae4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcee2ffd700 1 -- 172.21.15.53:0/3623187556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced80059a0 con 0x55d8a9ae61e0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcee37fe700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ae4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcee2ffd700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fced400f4c0 tx=0x7fced40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61401c216aeeb089 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcec1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4028040 con 0x55d8a9ae61e0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fced4014c80 con 0x55d8a9ae61e0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8a9aba3a0 con 0x55d8a9ae61e0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced4014e40 con 0x55d8a9ae61e0 2023-11-28T11:20:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcefce36b80 10 client.6177 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.526+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d8a98652b0 con 0x55d8a9ae61e0 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8a9aba3a0 con 0x55d8a9ae61e0 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fced4023440 con 0x55d8a9ae61e0 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 10 client.6177 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 10 client.6177 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 20 client.6177 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9abe8e0 0x55d8a9afa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 4 client.6177 mds_command: new command op to 55331 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55d8a99ce210 con 0x55d8a9abe8e0 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.527+0000 7fcee27fc700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9abe8e0 0x55d8a9afa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.528+0000 7fcec1ffb700 10 client.6177 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.528+0000 7fcee27fc700 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9abe8e0 0x55d8a9afa610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff745a6661e86d4c server_cookie=9a55fcc39d35c683 in_seq=0 out_seq=0 2023-11-28T11:20:58.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcec1ffb700 1 -- 172.21.15.53:0/3623187556 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d8a99ce210 con 0x55d8a9abe8e0 2023-11-28T11:20:58.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcec1ffb700 10 client.6177 handle_command_reply: tid=0 2023-11-28T11:20:58.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 client.6177 shutdown 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 msgr2=0x55d8a9b69100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fced400f4c0 tx=0x7fced40047c0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 shutdown_connections 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55d8a9abe8e0 0x55d8a9afa610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8a9ae91d0 0x55d8a9ae4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8a9ae61e0 0x55d8a9b69100 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 --2- 172.21.15.53:0/3623187556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d8a9ae88c0 0x55d8a9ae4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcec1ffb700 0 client.6177 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 >> 172.21.15.53:0/3623187556 conn(0x55d8a9abbe10 msgr2=0x55d8a9ad6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.534 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fa732bea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6135=0-4194304@1} caps={6135=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6135 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x558093d31600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:20:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 shutdown_connections 2023-11-28T11:20:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 1 -- 172.21.15.53:0/3623187556 wait complete. 2023-11-28T11:20:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.529+0000 7fcefce36b80 20 client.6177 trim_cache size 0 max 16384 2023-11-28T11:20:58.535 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:58.535 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2023-11-28T11:20:58.535 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "1e19a826-4f9f-4fa1-a7cd-da8de2b73c10", 2023-11-28T11:20:58.535 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2023-11-28T11:20:58.535 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 msgr2=0x7fced006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 0x7fced006a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcee4153de0 tx=0x7fced4008040 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 msgr2=0x7fcee40ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcedc014040 tx=0x7fcedc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 shutdown_connections 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fced00682f0 0x7fced006a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcee40cded0 0x7fcee41f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee415dc30 0x7fcee40ca240 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 --2- 172.21.15.53:0/3075956777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee415a760 0x7fcee415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 >> 172.21.15.53:0/3075956777 conn(0x7fcee4072620 msgr2=0x7fcee406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 shutdown_connections 2023-11-28T11:20:58.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.530+0000 7fcee95ff700 1 -- 172.21.15.53:0/3075956777 wait complete. 2023-11-28T11:20:58.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:20:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.751+0000 7f58033ff700 1 Processor -- start 2023-11-28T11:20:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.752+0000 7f58033ff700 1 -- start start 2023-11-28T11:20:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f58033ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57fc161340 0x7f57fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f58033ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57fc0d2320 0x7f57fc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f58033ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0d1950 con 0x7f57fc15c9b0 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f58033ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc0b5800 con 0x7f57fc0d2320 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f58033ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc0b5300 con 0x7f57fc161340 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f580199c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57fc0d2320 0x7f57fc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f580199c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57fc0d2320 0x7f57fc0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38426/0 (socket says 172.21.15.53:38426) 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f580199c700 1 -- 172.21.15.53:0/2089255058 learned_addr learned my addr 172.21.15.53:0/2089255058 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f580119b700 1 --1- 172.21.15.53:0/2089255058 >> v1:172.21.15.53:6789/0 conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36326/0 (socket says 172.21.15.53:36326) 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.753+0000 7f580099a700 1 --2- 172.21.15.53:0/2089255058 >> v2:172.21.15.157:3300/0 conn(0x7f57fc161340 0x7f57fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f580199c700 1 -- 172.21.15.53:0/2089255058 >> v2:172.21.15.157:3300/0 conn(0x7f57fc161340 msgr2=0x7f57fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f580199c700 1 --2- 172.21.15.53:0/2089255058 >> v2:172.21.15.157:3300/0 conn(0x7f57fc161340 0x7f57fc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f580199c700 1 -- 172.21.15.53:0/2089255058 >> v1:172.21.15.53:6789/0 conn(0x7f57fc15c9b0 legacy=0x7f57fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f580199c700 1 -- 172.21.15.53:0/2089255058 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0b5520 con 0x7f57fc0d2320 2023-11-28T11:20:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f57f3fff700 1 -- 172.21.15.53:0/2089255058 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112176525 0 0) 0x7f57fc0d1950 con 0x7f57fc15c9b0 2023-11-28T11:20:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f580199c700 1 --2- 172.21.15.53:0/2089255058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc0cd7c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0685c0 tx=0x7f57f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a88be3d84e8329 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f57f3fff700 1 -- 172.21.15.53:0/2089255058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400e070 con 0x7f57fc0d2320 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f57f3fff700 1 -- 172.21.15.53:0/2089255058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f400b8d0 con 0x7f57fc0d2320 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.754+0000 7f57f3fff700 1 -- 172.21.15.53:0/2089255058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400bc80 con 0x7f57fc0d2320 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 -- 172.21.15.53:0/2089255058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 msgr2=0x7f57fc0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 --2- 172.21.15.53:0/2089255058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc0cd7c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0685c0 tx=0x7f57f400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 -- 172.21.15.53:0/2089255058 shutdown_connections 2023-11-28T11:20:58.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 --2- 172.21.15.53:0/2089255058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc0cd7c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 --2- 172.21.15.53:0/2089255058 >> v2:172.21.15.157:3300/0 conn(0x7f57fc161340 0x7f57fc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 -- 172.21.15.53:0/2089255058 >> 172.21.15.53:0/2089255058 conn(0x7f57fc072600 msgr2=0x7f57fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 -- 172.21.15.53:0/2089255058 shutdown_connections 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.755+0000 7f58033ff700 1 -- 172.21.15.53:0/2089255058 wait complete. 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.757+0000 7f58033ff700 1 Processor -- start 2023-11-28T11:20:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.757+0000 7f58033ff700 1 -- start start 2023-11-28T11:20:58.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15c9b0 0x7f57fc154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57fc161340 0x7f57fc154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1565b0 con 0x7f57fc0d2320 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15e260 con 0x7f57fc15c9b0 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f58033ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc153b10 con 0x7f57fc161340 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38430/0 (socket says 172.21.15.53:38430) 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f580119b700 1 -- 172.21.15.53:0/1655045714 learned_addr learned my addr 172.21.15.53:0/1655045714 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f580099a700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15c9b0 0x7f57fc154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.758+0000 7f580199c700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57fc161340 0x7f57fc154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 -- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57fc161340 msgr2=0x7f57fc154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57fc161340 0x7f57fc154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 -- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15c9b0 msgr2=0x7f57fc154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15c9b0 0x7f57fc154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 -- 172.21.15.53:0/1655045714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009b30 con 0x7f57fc0d2320 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f580119b700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c550 tx=0x7f57f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f57f1ffb700 1 -- 172.21.15.53:0/1655045714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8004c80 con 0x7f57fc0d2320 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f57f1ffb700 1 -- 172.21.15.53:0/1655045714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8004e00 con 0x7f57fc0d2320 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc0d15e0 con 0x7f57fc0d2320 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f57f1ffb700 1 -- 172.21.15.53:0/1655045714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007310 con 0x7f57fc0d2320 2023-11-28T11:20:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.759+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0d1300 con 0x7f57fc0d2320 2023-11-28T11:20:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.762+0000 7f57f1ffb700 1 -- 172.21.15.53:0/1655045714 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57f801a020 con 0x7f57fc0d2320 2023-11-28T11:20:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.763+0000 7f57f1ffb700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 0x7f57e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.763+0000 7f57f1ffb700 1 -- 172.21.15.53:0/1655045714 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57f80857a0 con 0x7f57fc0d2320 2023-11-28T11:20:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.764+0000 7f580099a700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 0x7f57e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.764+0000 7f580099a700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 0x7f57e006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f57fc155720 tx=0x7f57ec009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.765+0000 7f5816e08b80 1 Processor -- start 2023-11-28T11:20:58.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.765+0000 7f5816e08b80 1 -- start start 2023-11-28T11:20:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da37dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da3997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da23dc10 con 0x5641da37d860 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da2ac3a0 con 0x5641da37e170 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641da2a58d0 con 0x5641da37b180 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38436/0 (socket says 172.21.15.53:38436) 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 -- 172.21.15.53:0/3531996267 learned_addr learned my addr 172.21.15.53:0/3531996267 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580099a700 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da37dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580199c700 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da3997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 -- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 msgr2=0x5641da37dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da37dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 -- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 msgr2=0x5641da3997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.766+0000 7f580119b700 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da3997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.767+0000 7f580119b700 1 -- 172.21.15.53:0/3531996267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641da127630 con 0x5641da37d860 2023-11-28T11:20:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.767+0000 7f580119b700 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f57f80557e0 tx=0x7f57f8055810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89638ec9a8dabb37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.767+0000 7f57e6ffd700 1 -- 172.21.15.53:0/3531996267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007670 con 0x5641da37d860 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.767+0000 7f57e6ffd700 1 -- 172.21.15.53:0/3531996267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f80550c0 con 0x5641da37d860 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.767+0000 7f57e6ffd700 1 -- 172.21.15.53:0/3531996267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f805d7d0 con 0x5641da37d860 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 -- 172.21.15.53:0/3531996267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 msgr2=0x5641da353a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f57f80557e0 tx=0x7f57f8055810 comp rx=0 tx=0).stop 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 -- 172.21.15.53:0/3531996267 shutdown_connections 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da3997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da353a70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 --2- 172.21.15.53:0/3531996267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da37dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 -- 172.21.15.53:0/3531996267 >> 172.21.15.53:0/3531996267 conn(0x5641da350db0 msgr2=0x5641da351740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 -- 172.21.15.53:0/3531996267 shutdown_connections 2023-11-28T11:20:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.768+0000 7f5816e08b80 1 -- 172.21.15.53:0/3531996267 wait complete. 2023-11-28T11:20:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.769+0000 7f5816e08b80 1 -- start start 2023-11-28T11:20:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.769+0000 7f5816e08b80 1 Processor -- start 2023-11-28T11:20:58.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da36e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da371bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da129920 con 0x5641da37d860 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da36fb70 con 0x5641da37e170 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641da372740 con 0x5641da37b180 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f5816e08b80 10 client.0 _ll_register_callbacks cb 0x5641da14b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38446/0 (socket says 172.21.15.53:38446) 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 -- 172.21.15.53:0/2161508995 learned_addr learned my addr 172.21.15.53:0/2161508995 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580119b700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da36e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580199c700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da371bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 -- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 msgr2=0x5641da36e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da36e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 -- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 msgr2=0x5641da371bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da371bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.771+0000 7f580099a700 1 -- 172.21.15.53:0/2161508995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0d1300 con 0x5641da37d860 2023-11-28T11:20:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f580099a700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f57ec011db0 tx=0x7f57ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b042115f41ffa618 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f57dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:58.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec01c040 con 0x5641da37d860 2023-11-28T11:20:58.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57ec003e10 con 0x5641da37d860 2023-11-28T11:20:58.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641da273190 con 0x5641da37d860 2023-11-28T11:20:58.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ec004590 con 0x5641da37d860 2023-11-28T11:20:58.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f5816e08b80 10 client.6192 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5641da0d3de0 con 0x5641da37d860 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.772+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641da273190 con 0x5641da37d860 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f57ec023450 con 0x5641da37d860 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 10 client.6192 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 10 client.6192 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 20 client.6192 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da36e810 0x5641da36ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 4 client.6192 mds_command: new command op to 55331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641da129f60 con 0x5641da36e810 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f580119b700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da36e810 0x5641da36ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f580119b700 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da36e810 0x5641da36ebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4544e386469631 server_cookie=8ff3c405d9c81af0 in_seq=0 out_seq=0 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.773+0000 7f57dbfff700 10 client.6192 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f57dbfff700 1 -- 172.21.15.53:0/2161508995 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5641da129f60 con 0x5641da36e810 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f57dbfff700 10 client.6192 handle_command_reply: tid=0 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 client.6192 shutdown 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 msgr2=0x5641da392470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f57ec011db0 tx=0x7f57ec004750 comp rx=0 tx=0).stop 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 shutdown_connections 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da36e810 0x5641da36ebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37b180 0x5641da36e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37e170 0x5641da371bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f57dbfff700 0 client.6192 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 --2- 172.21.15.53:0/2161508995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37d860 0x5641da392470 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.774+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 >> 172.21.15.53:0/2161508995 conn(0x5641da350db0 msgr2=0x5641da3539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.775+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 shutdown_connections 2023-11-28T11:20:58.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.775+0000 7f5816e08b80 1 -- 172.21.15.53:0/2161508995 wait complete. 2023-11-28T11:20:58.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.775+0000 7f5816e08b80 20 client.6192 trim_cache size 0 max 16384 2023-11-28T11:20:58.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.792+0000 7f5816e08b80 1 Processor -- start 2023-11-28T11:20:58.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.792+0000 7f5816e08b80 1 -- start start 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37b180 0x5641da3b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37d860 0x5641da39be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da3924e0 con 0x5641da37e170 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da392780 con 0x5641da37b180 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.793+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641da3ba710 con 0x5641da37d860 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38452/0 (socket says 172.21.15.53:38452) 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 -- 172.21.15.53:0/703561816 learned_addr learned my addr 172.21.15.53:0/703561816 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580119b700 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37b180 0x5641da3b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580099a700 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37d860 0x5641da39be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 -- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37d860 msgr2=0x5641da39be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37d860 0x5641da39be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 -- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37b180 msgr2=0x5641da3b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37b180 0x5641da3b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 -- 172.21.15.53:0/703561816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641da26c190 con 0x5641da37e170 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f580199c700 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b680 tx=0x7f57f4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb787d883914a4fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.794+0000 7f57e77fe700 1 -- 172.21.15.53:0/703561816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4022070 con 0x5641da37e170 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f57e77fe700 1 -- 172.21.15.53:0/703561816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f400bba0 con 0x5641da37e170 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f57e77fe700 1 -- 172.21.15.53:0/703561816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003c20 con 0x5641da37e170 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 -- 172.21.15.53:0/703561816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 msgr2=0x5641da36edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b680 tx=0x7f57f4010000 comp rx=0 tx=0).stop 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 -- 172.21.15.53:0/703561816 shutdown_connections 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37e170 0x5641da36edd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37b180 0x5641da3b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 --2- 172.21.15.53:0/703561816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37d860 0x5641da39be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 -- 172.21.15.53:0/703561816 >> 172.21.15.53:0/703561816 conn(0x5641da350db0 msgr2=0x5641da34de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 -- 172.21.15.53:0/703561816 shutdown_connections 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.795+0000 7f5816e08b80 1 -- 172.21.15.53:0/703561816 wait complete. 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.796+0000 7f5816e08b80 1 -- start start 2023-11-28T11:20:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.796+0000 7f5816e08b80 1 Processor -- start 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 0x5641da3f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37e170 0x5641da351420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da3e43c0 con 0x5641da37b180 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641da3e4b40 con 0x5641da37d860 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641da3b0970 con 0x5641da37e170 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f5816e08b80 10 client.0 _ll_register_callbacks cb 0x5641da14b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f580119b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38458/0 (socket says 172.21.15.53:38458) 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f580119b700 1 -- 172.21.15.53:0/277392079 learned_addr learned my addr 172.21.15.53:0/277392079 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:20:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f580099a700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 0x5641da3f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.798+0000 7f580199c700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37e170 0x5641da351420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 -- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37e170 msgr2=0x5641da351420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37e170 0x5641da351420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 -- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 msgr2=0x5641da3f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 0x5641da3f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 -- 172.21.15.53:0/277392079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0b5520 con 0x5641da37b180 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580099a700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 0x5641da3f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f580119b700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f57f8055b00 tx=0x7f57f8058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c661f188f6ec4895 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f57dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8063230 con 0x5641da37b180 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57f8063070 con 0x5641da37b180 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641da2631b0 con 0x5641da37b180 2023-11-28T11:20:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f8063390 con 0x5641da37b180 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f5816e08b80 10 client.6198 fetch_fsmap learned FSMap version 32 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.799+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5641da0fa250 con 0x5641da37b180 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641da2631b0 con 0x5641da37b180 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f57f8072080 con 0x5641da37b180 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 10 client.6198 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 10 client.6198 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 20 client.6198 populate_metadata read hostname 'smithi053' 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da351960 0x5641da375950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 4 client.6198 mds_command: new command op to 55331 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5641da24d1c0 con 0x5641da351960 2023-11-28T11:20:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f580099a700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da351960 0x5641da375950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f57dbfff700 10 client.6198 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.800+0000 7f580099a700 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da351960 0x5641da375950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f32cf3424b93b56 server_cookie=2ce771f9ea3a91a0 in_seq=0 out_seq=0 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f57dbfff700 1 -- 172.21.15.53:0/277392079 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x5641da24d1c0 con 0x5641da351960 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f57dbfff700 10 client.6198 handle_command_reply: tid=0 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 client.6198 shutdown 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 msgr2=0x5641da3e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f57f8055b00 tx=0x7f57f8058040 comp rx=0 tx=0).stop 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 shutdown_connections 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5641da351960 0x5641da375950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5641da37e170 0x5641da351420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641da37b180 0x5641da3e3e80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 --2- 172.21.15.53:0/277392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641da37d860 0x5641da3f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 >> 172.21.15.53:0/277392079 conn(0x5641da350db0 msgr2=0x5641da34de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f57dbfff700 0 client.6198 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 shutdown_connections 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 1 -- 172.21.15.53:0/277392079 wait complete. 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.801+0000 7f5816e08b80 20 client.6198 trim_cache size 0 max 16384 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:20:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "1e19a826-4f9f-4fa1-a7cd-da8de2b73c10": { 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "1e19a826-4f9f-4fa1-a7cd-da8de2b73c10", 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive" 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:20:58.807 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:20:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 msgr2=0x7f57e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 0x7f57e006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f57fc155720 tx=0x7f57ec009210 comp rx=0 tx=0).stop 2023-11-28T11:20:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 msgr2=0x7f57fc1f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c550 tx=0x7f57f800c860 comp rx=0 tx=0).stop 2023-11-28T11:20:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 shutdown_connections 2023-11-28T11:20:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f57e00682f0 0x7f57e006a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57fc161340 0x7f57fc154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc0d2320 0x7f57fc1f2a90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 --2- 172.21.15.53:0/1655045714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15c9b0 0x7f57fc154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 >> 172.21.15.53:0/1655045714 conn(0x7f57fc072600 msgr2=0x7f57fc15b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 shutdown_connections 2023-11-28T11:20:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:20:58.802+0000 7f58033ff700 1 -- 172.21.15.53:0/1655045714 wait complete. 2023-11-28T11:20:58.823 INFO:tasks.cephfs.filesystem:scrub status for tag:1e19a826-4f9f-4fa1-a7cd-da8de2b73c10 - {'path': '/', 'tag': '1e19a826-4f9f-4fa1-a7cd-da8de2b73c10', 'options': 'recursive'} 2023-11-28T11:21:04.863 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-11-28T11:21:04.862+0000 7fa6ddbf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:21:15.154 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:15.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:28.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:21:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.037+0000 7f0d655ff700 1 Processor -- start 2023-11-28T11:21:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.038+0000 7f0d655ff700 1 -- start start 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d655ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d60161340 0x7f0d600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d655ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d600d1950 con 0x7f0d6015c9b0 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d655ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d600b5520 con 0x7f0d600d2320 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d655ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d600b5800 con 0x7f0d60161340 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d5effd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0d6015c9b0 0x7f0d600b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56766/0 (socket says 172.21.15.53:56766) 2023-11-28T11:21:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d5effd700 1 -- 172.21.15.53:0/34844950 learned_addr learned my addr 172.21.15.53:0/34844950 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d57fff700 1 --2- 172.21.15.53:0/34844950 >> v2:172.21.15.157:3300/0 conn(0x7f0d60161340 0x7f0d600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.039+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994603651 0 0) 0x7f0d600d1950 con 0x7f0d6015c9b0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d4c0035b0 con 0x7f0d6015c9b0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890590486 0 0) 0x7f0d4c0035b0 con 0x7f0d6015c9b0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 >> v2:172.21.15.157:3300/0 conn(0x7f0d60161340 msgr2=0x7f0d600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 --2- 172.21.15.53:0/34844950 >> v2:172.21.15.157:3300/0 conn(0x7f0d60161340 0x7f0d600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 >> v1:172.21.15.157:6789/0 conn(0x7f0d600d2320 legacy=0x7f0d600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600d1300 con 0x7f0d6015c9b0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f0d50001670 con 0x7f0d6015c9b0 2023-11-28T11:21:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 >> v1:172.21.15.53:6789/0 conn(0x7f0d6015c9b0 legacy=0x7f0d600b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 0x7f0d4c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c004850 0x7f0d4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c005250 0x7f0d4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600b5800 con 0x7f0d4c003a50 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c00c160 con 0x7f0d4c005250 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c00c320 con 0x7f0d4c004850 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.040+0000 7f0d5effd700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 0x7f0d4c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d57fff700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c004850 0x7f0d4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c005250 0x7f0d4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 -- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c004850 msgr2=0x7f0d4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c004850 0x7f0d4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 -- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c005250 msgr2=0x7f0d4c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c005250 0x7f0d4c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 -- 172.21.15.53:0/34844950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0035b0 con 0x7f0d4c003a50 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d57fff700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c004850 0x7f0d4c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c005250 0x7f0d4c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/34844950 reap_dead start 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5effd700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 0x7f0d4c003e60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c160 tx=0x7f0d4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a9c7491049ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50007b70 con 0x7f0d4c003a50 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000cd20 con 0x7f0d4c003a50 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.041+0000 7f0d5e7fc700 1 -- 172.21.15.53:0/34844950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50005ba0 con 0x7f0d4c003a50 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 -- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 msgr2=0x7f0d4c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 0x7f0d4c003e60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c160 tx=0x7f0d4c005880 comp rx=0 tx=0).stop 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 -- 172.21.15.53:0/34844950 shutdown_connections 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 --2- 172.21.15.53:0/34844950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d4c003a50 0x7f0d4c003e60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 -- 172.21.15.53:0/34844950 >> 172.21.15.53:0/34844950 conn(0x7f0d60072600 msgr2=0x7f0d600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 -- 172.21.15.53:0/34844950 shutdown_connections 2023-11-28T11:21:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.042+0000 7f0d655ff700 1 -- 172.21.15.53:0/34844950 wait complete. 2023-11-28T11:21:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.044+0000 7f0d655ff700 1 Processor -- start 2023-11-28T11:21:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.044+0000 7f0d655ff700 1 -- start start 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 0x7f0d600cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d601e8890 0x7f0d60156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600c7490 con 0x7f0d6015e150 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601e9010 con 0x7f0d4c003a50 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d655ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6015eb10 con 0x7f0d601e8890 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40050/0 (socket says 172.21.15.53:40050) 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/248177392 learned_addr learned my addr 172.21.15.53:0/248177392 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d57fff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d601e8890 0x7f0d60156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5effd700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 0x7f0d600cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d601e8890 msgr2=0x7f0d60156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d601e8890 0x7f0d60156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 msgr2=0x7f0d600cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 0x7f0d600cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/248177392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6015cd80 con 0x7f0d6015e150 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.045+0000 7f0d5effd700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 0x7f0d600cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0d600cfbc0 tx=0x7f0d5800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d577fe700 1 -- 172.21.15.53:0/248177392 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58003e10 con 0x7f0d6015e150 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d577fe700 1 -- 172.21.15.53:0/248177392 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d58005070 con 0x7f0d6015e150 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d600d15e0 con 0x7f0d6015e150 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d577fe700 1 -- 172.21.15.53:0/248177392 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58005420 con 0x7f0d6015e150 2023-11-28T11:21:29.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.046+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6015cf10 con 0x7f0d6015e150 2023-11-28T11:21:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.048+0000 7f0d577fe700 1 -- 172.21.15.53:0/248177392 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d5801a030 con 0x7f0d6015e150 2023-11-28T11:21:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.049+0000 7f0d577fe700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 0x7f0d4006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.049+0000 7f0d577fe700 1 -- 172.21.15.53:0/248177392 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d580857c0 con 0x7f0d6015e150 2023-11-28T11:21:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.049+0000 7f0d5effd700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 0x7f0d4006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.050+0000 7f0d5effd700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 0x7f0d4006a7d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0d50015040 tx=0x7f0d50007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.051+0000 7f0d78f11b80 1 Processor -- start 2023-11-28T11:21:29.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.051+0000 7f0d78f11b80 1 -- start start 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efce7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efbd1c10 con 0x5646efd12170 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efc403a0 con 0x5646efd0f180 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646efc398d0 con 0x5646efd11860 2023-11-28T11:21:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40052/0 (socket says 172.21.15.53:40052) 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.052+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/3932063520 learned_addr learned my addr 172.21.15.53:0/3932063520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5effd700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efce7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 msgr2=0x5646efce7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efce7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 msgr2=0x5646efd11c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd11c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 -- 172.21.15.53:0/3932063520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646efabb630 con 0x5646efd12170 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d580555d0 tx=0x7f0d58055600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4106d868ccb95101 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d54ff9700 1 -- 172.21.15.53:0/3932063520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5805d7f0 con 0x5646efd12170 2023-11-28T11:21:29.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d54ff9700 1 -- 172.21.15.53:0/3932063520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5805d950 con 0x5646efd12170 2023-11-28T11:21:29.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.053+0000 7f0d54ff9700 1 -- 172.21.15.53:0/3932063520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5805dac0 con 0x5646efd12170 2023-11-28T11:21:29.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3932063520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 msgr2=0x5646efd2d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d580555d0 tx=0x7f0d58055600 comp rx=0 tx=0).stop 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3932063520 shutdown_connections 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efce7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd12170 0x5646efd2d7c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3932063520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3932063520 >> 172.21.15.53:0/3932063520 conn(0x5646efce4db0 msgr2=0x5646efce5740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3932063520 shutdown_connections 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.054+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3932063520 wait complete. 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.055+0000 7f0d78f11b80 1 -- start start 2023-11-28T11:21:29.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.055+0000 7f0d78f11b80 1 Processor -- start 2023-11-28T11:21:29.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efd14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 0x5646efd14f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efabd920 con 0x5646efd0f180 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efd07ca0 con 0x5646efd12170 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646efd04a30 con 0x5646efd11860 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d78f11b80 10 client.0 _ll_register_callbacks cb 0x5646efadf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40060/0 (socket says 172.21.15.53:40060) 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d5effd700 1 -- 172.21.15.53:0/3455898061 learned_addr learned my addr 172.21.15.53:0/3455898061 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efd14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.057+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 0x5646efd14f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 -- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 msgr2=0x5646efd14a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efd14a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 -- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 msgr2=0x5646efd14f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 0x5646efd14f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 -- 172.21.15.53:0/3455898061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6015cf10 con 0x5646efd0f180 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 0x5646efd14f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efd14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d5effd700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c130 tx=0x7f0d50009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d5698be4b539a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d500075a0 con 0x5646efd0f180 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d50007700 con 0x5646efd0f180 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646efc07190 con 0x5646efd0f180 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d50032420 con 0x5646efd0f180 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.058+0000 7f0d78f11b80 10 client.6210 fetch_fsmap learned FSMap version 33 2023-11-28T11:21:29.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5646efa67de0 con 0x5646efd0f180 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50038070 con 0x5646efd0f180 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0d50032580 con 0x5646efd0f180 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 10 client.6210 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 10 client.6210 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 20 client.6210 populate_metadata read hostname 'smithi053' 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efd04bb0 0x5646efd055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 4 client.6210 mds_command: new command op to 55331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646efabdf60 con 0x5646efd04bb0 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.059+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efd04bb0 0x5646efd055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.060+0000 7f0d57fff700 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efd04bb0 0x5646efd055f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1610ade8ea056aa server_cookie=14f9b3fcd748d54d in_seq=0 out_seq=0 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.060+0000 7f0d3dffb700 10 client.6210 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.060+0000 7f0d3dffb700 1 -- 172.21.15.53:0/3455898061 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5646efabdf60 con 0x5646efd04bb0 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.060+0000 7f0d3dffb700 10 client.6210 handle_command_reply: tid=0 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.060+0000 7f0d78f11b80 1 client.6210 shutdown 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 msgr2=0x5646efd044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c130 tx=0x7f0d50009050 comp rx=0 tx=0).stop 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 shutdown_connections 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efd04bb0 0x5646efd055f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd12170 0x5646efd14f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd0f180 0x5646efd044f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/3455898061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd11860 0x5646efd14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d3dffb700 0 client.6210 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 >> 172.21.15.53:0/3455898061 conn(0x5646efce4db0 msgr2=0x5646efce79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 shutdown_connections 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 1 -- 172.21.15.53:0/3455898061 wait complete. 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.061+0000 7f0d78f11b80 20 client.6210 trim_cache size 0 max 16384 2023-11-28T11:21:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.078+0000 7f0d78f11b80 1 Processor -- start 2023-11-28T11:21:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.078+0000 7f0d78f11b80 1 -- start start 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efce5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efce0e40 con 0x5646efd11860 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efd25540 con 0x5646efd0f180 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.079+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646efce0640 con 0x5646efd12170 2023-11-28T11:21:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40064/0 (socket says 172.21.15.53:40064) 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 -- 172.21.15.53:0/1601021456 learned_addr learned my addr 172.21.15.53:0/1601021456 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d5effd700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efce5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 -- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 msgr2=0x5646efd14eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 -- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 msgr2=0x5646efce5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efce5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 -- 172.21.15.53:0/1601021456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646efc00190 con 0x5646efd11860 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d5f7fe700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d6004fef0 tx=0x7f0d48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7573873d919b50ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d557fa700 1 -- 172.21.15.53:0/1601021456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4801c070 con 0x5646efd11860 2023-11-28T11:21:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d557fa700 1 -- 172.21.15.53:0/1601021456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4801caa0 con 0x5646efd11860 2023-11-28T11:21:29.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.080+0000 7f0d557fa700 1 -- 172.21.15.53:0/1601021456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4801ce50 con 0x5646efd11860 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1601021456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 msgr2=0x5646efd14970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d6004fef0 tx=0x7f0d48008040 comp rx=0 tx=0).stop 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1601021456 shutdown_connections 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efce5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1601021456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1601021456 >> 172.21.15.53:0/1601021456 conn(0x5646efce4db0 msgr2=0x5646efd03830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1601021456 shutdown_connections 2023-11-28T11:21:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.081+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1601021456 wait complete. 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.082+0000 7f0d78f11b80 1 -- start start 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.082+0000 7f0d78f11b80 1 Processor -- start 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd91a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efd921a0 con 0x5646efd11860 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646efd22480 con 0x5646efd0f180 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646efd91f50 con 0x5646efd12170 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d78f11b80 10 client.0 _ll_register_callbacks cb 0x5646efadf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40066/0 (socket says 172.21.15.53:40066) 2023-11-28T11:21:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.083+0000 7f0d57fff700 1 -- 172.21.15.53:0/1486031876 learned_addr learned my addr 172.21.15.53:0/1486031876 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 -- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 msgr2=0x5646efd14eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 -- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 msgr2=0x5646efd91a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d5effd700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd91a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd91a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 -- 172.21.15.53:0/1486031876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646efa8e250 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d5effd700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd91a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d57fff700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d48000c00 tx=0x7f0d5000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a5b9bb8df55ce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4801ce10 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d48005a20 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646efbf71b0 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d48005c20 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d78f11b80 10 client.6216 fetch_fsmap learned FSMap version 33 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.084+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5646efa50c60 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646efbf71b0 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0d48016430 con 0x5646efd11860 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 10 client.6216 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:21:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 10 client.6216 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 20 client.6216 populate_metadata read hostname 'smithi053' 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efce6870 0x5646efd12a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 4 client.6216 mds_command: new command op to 55331 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5646efbe11c0 con 0x5646efce6870 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d5effd700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efce6870 0x5646efd12a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d3dffb700 10 client.6216 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.085+0000 7f0d5effd700 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efce6870 0x5646efd12a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2ba9373a3766b4 server_cookie=96a7c42d65e463bb in_seq=0 out_seq=0 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d3dffb700 1 -- 172.21.15.53:0/1486031876 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5646efbe11c0 con 0x5646efce6870 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d3dffb700 10 client.6216 handle_command_reply: tid=0 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 client.6216 shutdown 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 msgr2=0x5646efd14970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d48000c00 tx=0x7f0d5000a000 comp rx=0 tx=0).stop 2023-11-28T11:21:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 shutdown_connections 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x5646efce6870 0x5646efd12a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5646efd12170 0x5646efd14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646efd0f180 0x5646efd91a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 --2- 172.21.15.53:0/1486031876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646efd11860 0x5646efd14970 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d3dffb700 0 client.6216 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 >> 172.21.15.53:0/1486031876 conn(0x5646efce4db0 msgr2=0x5646efd0e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 shutdown_connections 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 1 -- 172.21.15.53:0/1486031876 wait complete. 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.086+0000 7f0d78f11b80 20 client.6216 trim_cache size 0 max 16384 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:21:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 msgr2=0x7f0d4006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 0x7f0d4006a7d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0d50015040 tx=0x7f0d50007000 comp rx=0 tx=0).stop 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 msgr2=0x7f0d6015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0d600cfbc0 tx=0x7f0d5800de00 comp rx=0 tx=0).stop 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 shutdown_connections 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f0d40068320 0x7f0d4006a7d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6015e150 0x7f0d6015e5d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c003a50 0x7f0d600cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 --2- 172.21.15.53:0/248177392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d601e8890 0x7f0d60156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 >> 172.21.15.53:0/248177392 conn(0x7f0d60072600 msgr2=0x7f0d6015a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 shutdown_connections 2023-11-28T11:21:29.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.087+0000 7f0d655ff700 1 -- 172.21.15.53:0/248177392 wait complete. 2023-11-28T11:21:29.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:21:29.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.306+0000 7feec97ff700 1 Processor -- start 2023-11-28T11:21:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.306+0000 7feec97ff700 1 -- start start 2023-11-28T11:21:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec97ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec97ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec40d1950 con 0x7feec415c9b0 2023-11-28T11:21:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec97ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec40b5520 con 0x7feec40d2320 2023-11-28T11:21:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec97ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40b5800 con 0x7feec4161340 2023-11-28T11:21:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec2ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feec415c9b0 0x7feec40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56780/0 (socket says 172.21.15.53:56780) 2023-11-28T11:21:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.307+0000 7feec2ffd700 1 -- 172.21.15.53:0/271281227 learned_addr learned my addr 172.21.15.53:0/271281227 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec27fc700 1 --2- 172.21.15.53:0/271281227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316802861 0 0) 0x7feec40d1950 con 0x7feec415c9b0 2023-11-28T11:21:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeac0035b0 con 0x7feec415c9b0 2023-11-28T11:21:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728361288 0 0) 0x7feec40b5520 con 0x7feec40d2320 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec40d1950 con 0x7feec40d2320 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec27fc700 1 -- 172.21.15.53:0/271281227 >> v1:172.21.15.157:6789/0 conn(0x7feec40d2320 legacy=0x7feec40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec27fc700 1 -- 172.21.15.53:0/271281227 >> v1:172.21.15.53:6789/0 conn(0x7feec415c9b0 legacy=0x7feec40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec27fc700 1 -- 172.21.15.53:0/271281227 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40d1300 con 0x7feec4161340 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.308+0000 7feec27fc700 1 --2- 172.21.15.53:0/271281227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40d1de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2094ef13a1a30b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8011830 con 0x7feec4161340 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8011990 con 0x7feec4161340 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec1ffb700 1 -- 172.21.15.53:0/271281227 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8011d20 con 0x7feec4161340 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 -- 172.21.15.53:0/271281227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 msgr2=0x7feec40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 --2- 172.21.15.53:0/271281227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40d1de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb8009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 -- 172.21.15.53:0/271281227 shutdown_connections 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 --2- 172.21.15.53:0/271281227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40d1de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 -- 172.21.15.53:0/271281227 >> 172.21.15.53:0/271281227 conn(0x7feec4072600 msgr2=0x7feec40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.309+0000 7feec97ff700 1 -- 172.21.15.53:0/271281227 shutdown_connections 2023-11-28T11:21:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.310+0000 7feec97ff700 1 -- 172.21.15.53:0/271281227 wait complete. 2023-11-28T11:21:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.311+0000 7feec97ff700 1 Processor -- start 2023-11-28T11:21:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.311+0000 7feec97ff700 1 -- start start 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40d2320 0x7feec40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40cb590 con 0x7feec415c9b0 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4154110 con 0x7feec40d2320 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec97ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40cac60 con 0x7feec4161340 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.312+0000 7feec27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40078/0 (socket says 172.21.15.53:40078) 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 -- 172.21.15.53:0/1382038215 learned_addr learned my addr 172.21.15.53:0/1382038215 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec2ffd700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40d2320 0x7feec40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec37fe700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 -- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 msgr2=0x7feec40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 -- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40d2320 msgr2=0x7feec40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40d2320 0x7feec40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 -- 172.21.15.53:0/1382038215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80094a0 con 0x7feec415c9b0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feec27fc700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097c0 tx=0x7feeb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb801a970 con 0x7feec415c9b0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb801aad0 con 0x7feec415c9b0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.313+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb801ae40 con 0x7feec415c9b0 2023-11-28T11:21:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.314+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec415cd80 con 0x7feec415c9b0 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.314+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec40d15e0 con 0x7feec415c9b0 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.315+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec415cf10 con 0x7feec415c9b0 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.316+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feeb800f050 con 0x7feec415c9b0 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.316+0000 7feeab7fe700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 0x7feeb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.316+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feeb80870c0 con 0x7feec415c9b0 2023-11-28T11:21:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.317+0000 7feec2ffd700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 0x7feeb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.318+0000 7feec2ffd700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 0x7feeb006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007fd0 tx=0x7feeb4007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.321+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feeb8053830 con 0x7feec415c9b0 2023-11-28T11:21:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.569+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec40cb740 con 0x7feec415c9b0 2023-11-28T11:21:29.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.570+0000 7feeab7fe700 1 -- 172.21.15.53:0/1382038215 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4659 (secure 0 0 0) 0x7feec40cb740 con 0x7feec415c9b0 2023-11-28T11:21:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:29.571 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:21:04.863310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:21:29.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 msgr2=0x7feeb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 0x7feeb006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007fd0 tx=0x7feeb4007e00 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 msgr2=0x7feec4153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097c0 tx=0x7feeb8004750 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 shutdown_connections 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feeb00682f0 0x7feeb006a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feec4161340 0x7feec40c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40d2320 0x7feec40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 --2- 172.21.15.53:0/1382038215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec415c9b0 0x7feec4153bd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 >> 172.21.15.53:0/1382038215 conn(0x7feec4072600 msgr2=0x7feec406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 shutdown_connections 2023-11-28T11:21:29.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.572+0000 7feec97ff700 1 -- 172.21.15.53:0/1382038215 wait complete. 2023-11-28T11:21:29.575 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 33 2023-11-28T11:21:29.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-28T11:21:29.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.795+0000 7f5d5b9ff700 1 Processor -- start 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.795+0000 7f5d5b9ff700 1 -- start start 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d5b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d54161340 0x7f5d540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d5b9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d540d2320 0x7f5d540cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d5b9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540d1950 con 0x7f5d5415c9b0 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d5b9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540b5800 con 0x7f5d540d2320 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d5b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540b5300 con 0x7f5d54161340 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d59f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d540d2320 0x7f5d540cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d59f9c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d540d2320 0x7f5d540cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40082/0 (socket says 172.21.15.53:40082) 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d59f9c700 1 -- 172.21.15.53:0/2696197006 learned_addr learned my addr 172.21.15.53:0/2696197006 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.796+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d54161340 0x7f5d540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d59f9c700 1 -- 172.21.15.53:0/2696197006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d54161340 msgr2=0x7f5d540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d54161340 0x7f5d540d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d4bfff700 1 -- 172.21.15.53:0/2696197006 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247079834 0 0) 0x7f5d540d1950 con 0x7f5d5415c9b0 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d59f9c700 1 -- 172.21.15.53:0/2696197006 >> v1:172.21.15.157:6789/0 conn(0x7f5d5415c9b0 legacy=0x7f5d540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d59f9c700 1 -- 172.21.15.53:0/2696197006 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540b5520 con 0x7f5d540d2320 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d540d2320 0x7f5d540cd7c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e80 tx=0x7f5d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1da509d205e6801 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d4bfff700 1 -- 172.21.15.53:0/2696197006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00e070 con 0x7f5d540d2320 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d4bfff700 1 -- 172.21.15.53:0/2696197006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00b8d0 con 0x7f5d540d2320 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.797+0000 7f5d4bfff700 1 -- 172.21.15.53:0/2696197006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00bc80 con 0x7f5d540d2320 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/2696197006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d540d2320 msgr2=0x7f5d540cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d540d2320 0x7f5d540cd7c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e80 tx=0x7f5d4c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/2696197006 shutdown_connections 2023-11-28T11:21:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d540d2320 0x7f5d540cd7c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/2696197006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d54161340 0x7f5d540d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/2696197006 >> 172.21.15.53:0/2696197006 conn(0x7f5d54072600 msgr2=0x7f5d540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/2696197006 shutdown_connections 2023-11-28T11:21:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.798+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/2696197006 wait complete. 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.800+0000 7f5d5b9ff700 1 Processor -- start 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.800+0000 7f5d5b9ff700 1 -- start start 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540d2320 0x7f5d541e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5415c9b0 0x7f5d5414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541e8f60 con 0x7f5d54161340 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541e8350 con 0x7f5d540d2320 2023-11-28T11:21:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541e8630 con 0x7f5d5415c9b0 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40092/0 (socket says 172.21.15.53:40092) 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 -- 172.21.15.53:0/184327341 learned_addr learned my addr 172.21.15.53:0/184327341 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5415c9b0 0x7f5d5414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d5979b700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540d2320 0x7f5d541e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 -- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5415c9b0 msgr2=0x7f5d5414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5415c9b0 0x7f5d5414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 -- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540d2320 msgr2=0x7f5d541e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540d2320 0x7f5d541e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.801+0000 7f5d59f9c700 1 -- 172.21.15.53:0/184327341 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009b30 con 0x7f5d54161340 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b680 tx=0x7f5d4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d49ffb700 1 -- 172.21.15.53:0/184327341 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01b630 con 0x7f5d54161340 2023-11-28T11:21:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d49ffb700 1 -- 172.21.15.53:0/184327341 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c01b7b0 con 0x7f5d54161340 2023-11-28T11:21:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d49ffb700 1 -- 172.21.15.53:0/184327341 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01bb60 con 0x7f5d54161340 2023-11-28T11:21:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540d15e0 con 0x7f5d54161340 2023-11-28T11:21:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.802+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540d1300 con 0x7f5d54161340 2023-11-28T11:21:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.804+0000 7f5d49ffb700 1 -- 172.21.15.53:0/184327341 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d4c023030 con 0x7f5d54161340 2023-11-28T11:21:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.805+0000 7f5d49ffb700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 0x7f5d3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.805+0000 7f5d49ffb700 1 -- 172.21.15.53:0/184327341 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d4c0871d0 con 0x7f5d54161340 2023-11-28T11:21:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.805+0000 7f5d5979b700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 0x7f5d3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.806+0000 7f5d5979b700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 0x7f5d3806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000f60 tx=0x7f5d50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.807+0000 7f5d6f280b80 1 Processor -- start 2023-11-28T11:21:29.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.807+0000 7f5d6f280b80 1 -- start start 2023-11-28T11:21:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972271ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c9722b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97215bc60 con 0x55c9722991d0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9721ca3f0 con 0x55c97229b8b0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9721c3920 con 0x55c97229c1c0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d58f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d58f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40106/0 (socket says 172.21.15.53:40106) 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d58f9a700 1 -- 172.21.15.53:0/2300799450 learned_addr learned my addr 172.21.15.53:0/2300799450 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d5979b700 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972271ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.808+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c9722b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 -- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 msgr2=0x55c9722b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c9722b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 -- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 msgr2=0x55c972271ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972271ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 -- 172.21.15.53:0/2300799450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c972045680 con 0x55c9722991d0 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009cc0 tx=0x7f5d440112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38ac32887fd7c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d36ffd700 1 -- 172.21.15.53:0/2300799450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44004300 con 0x55c9722991d0 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d36ffd700 1 -- 172.21.15.53:0/2300799450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44004460 con 0x55c9722991d0 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.809+0000 7f5d36ffd700 1 -- 172.21.15.53:0/2300799450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44011b30 con 0x55c9722991d0 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2300799450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 msgr2=0x55c97229bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009cc0 tx=0x7f5d440112c0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2300799450 shutdown_connections 2023-11-28T11:21:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c9722b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972271ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2300799450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229bc80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2300799450 >> 172.21.15.53:0/2300799450 conn(0x55c97226ee00 msgr2=0x55c97226f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2300799450 shutdown_connections 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.810+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2300799450 wait complete. 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.811+0000 7f5d6f280b80 1 -- start start 2023-11-28T11:21:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.811+0000 7f5d6f280b80 1 Processor -- start 2023-11-28T11:21:29.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229b8b0 0x55c9722937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229c1c0 0x55c97226af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972047970 con 0x55c9722991d0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9722bae20 con 0x55c97229c1c0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c972294f50 con 0x55c97229b8b0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d6f280b80 10 client.0 _ll_register_callbacks cb 0x55c9720692e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d5979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d5979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40116/0 (socket says 172.21.15.53:40116) 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d5979b700 1 -- 172.21.15.53:0/2458720029 learned_addr learned my addr 172.21.15.53:0/2458720029 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.813+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229b8b0 0x55c9722937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229c1c0 0x55c97226af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 -- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229b8b0 msgr2=0x55c9722937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229b8b0 0x55c9722937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 -- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229c1c0 msgr2=0x55c97226af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229c1c0 0x55c97226af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 -- 172.21.15.53:0/2458720029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d440098c0 con 0x55c9722991d0 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d5979b700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5d50014480 tx=0x7f5d50001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a71430dda0f51e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:21:29.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50001d50 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d5001c040 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.814+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9721911e0 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d50014d70 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 10 client.6234 fetch_fsmap learned FSMap version 33 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55c971ff1e30 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c9721911e0 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5d50021400 con 0x55c9722991d0 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 10 client.6234 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 10 client.6234 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 20 client.6234 populate_metadata read hostname 'smithi053' 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c972293ec0 0x55c97229e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 4 client.6234 mds_command: new command op to 55331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c972047fb0 con 0x55c972293ec0 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.815+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c972293ec0 0x55c97229e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.816+0000 7f5d2ffff700 10 client.6234 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.816+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c972293ec0 0x55c97229e280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5092e60e7657fde server_cookie=d0154c4c0c3657e6 in_seq=0 out_seq=0 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.816+0000 7f5d2ffff700 1 -- 172.21.15.53:0/2458720029 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c972047fb0 con 0x55c972293ec0 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.816+0000 7f5d2ffff700 10 client.6234 handle_command_reply: tid=0 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.816+0000 7f5d6f280b80 1 client.6234 shutdown 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 msgr2=0x55c97229f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5d50014480 tx=0x7f5d50001080 comp rx=0 tx=0).stop 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 shutdown_connections 2023-11-28T11:21:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c972293ec0 0x55c97229e280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229c1c0 0x55c97226af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c97229f630 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d2ffff700 0 client.6234 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/2458720029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229b8b0 0x55c9722937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 >> 172.21.15.53:0/2458720029 conn(0x55c97226ee00 msgr2=0x55c972271a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 shutdown_connections 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 1 -- 172.21.15.53:0/2458720029 wait complete. 2023-11-28T11:21:29.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.817+0000 7f5d6f280b80 20 client.6234 trim_cache size 0 max 16384 2023-11-28T11:21:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.830+0000 7f5d6f280b80 1 Processor -- start 2023-11-28T11:21:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 -- start start 2023-11-28T11:21:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 0x55c97226d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972270da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972272110 con 0x55c97229c1c0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97229c030 con 0x55c97229b8b0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.831+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c97226b030 con 0x55c9722991d0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40118/0 (socket says 172.21.15.53:40118) 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 -- 172.21.15.53:0/3029431958 learned_addr learned my addr 172.21.15.53:0/3029431958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972270da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d5979b700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 0x55c97226d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 -- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 msgr2=0x55c97226d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 0x55c97226d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 -- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 msgr2=0x55c972270da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972270da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 -- 172.21.15.53:0/3029431958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c97218a1e0 con 0x55c97229c1c0 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972270da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d5979b700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 0x55c97226d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d59f9c700 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c008e50 tx=0x7f5d4c087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c6fd5c0b2db3b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.832+0000 7f5d377fe700 1 -- 172.21.15.53:0/3029431958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c02d300 con 0x55c97229c1c0 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d377fe700 1 -- 172.21.15.53:0/3029431958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c05f690 con 0x55c97229c1c0 2023-11-28T11:21:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d377fe700 1 -- 172.21.15.53:0/3029431958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0092e0 con 0x55c97229c1c0 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 -- 172.21.15.53:0/3029431958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 msgr2=0x55c972289520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c008e50 tx=0x7f5d4c087050 comp rx=0 tx=0).stop 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 -- 172.21.15.53:0/3029431958 shutdown_connections 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c9722991d0 0x55c97226d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c97229c1c0 0x55c972289520 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/3029431958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c972270da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 -- 172.21.15.53:0/3029431958 >> 172.21.15.53:0/3029431958 conn(0x55c97226ee00 msgr2=0x55c972271880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 -- 172.21.15.53:0/3029431958 shutdown_connections 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.833+0000 7f5d6f280b80 1 -- 172.21.15.53:0/3029431958 wait complete. 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.834+0000 7f5d6f280b80 1 -- start start 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.834+0000 7f5d6f280b80 1 Processor -- start 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c97231beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c972271920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97231c9b0 con 0x55c9722991d0 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9722ba3d0 con 0x55c97229b8b0 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c972287510 con 0x55c97229c1c0 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d6f280b80 10 client.0 _ll_register_callbacks cb 0x55c9720692e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d5979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d5979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40124/0 (socket says 172.21.15.53:40124) 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.835+0000 7f5d5979b700 1 -- 172.21.15.53:0/914193252 learned_addr learned my addr 172.21.15.53:0/914193252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 -- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 msgr2=0x55c972271920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c972271920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c97231beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 -- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 msgr2=0x55c97231beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c97231beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 -- 172.21.15.53:0/914193252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540d1300 con 0x55c9722991d0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c97231beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d5979b700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5d5002aa10 tx=0x7f5d50015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cac7225a869db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50000f90 con 0x55c9722991d0 2023-11-28T11:21:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d50021400 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c972181200 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5000f4b0 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d6f280b80 10 client.6240 fetch_fsmap learned FSMap version 33 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.836+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55c9720182a0 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c972181200 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5d50021910 con 0x55c9722991d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 10 client.6240 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 10 client.6240 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 20 client.6240 populate_metadata read hostname 'smithi053' 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c9722898d0 0x55c972289ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 4 client.6240 mds_command: new command op to 55331 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 --> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55c97216b210 con 0x55c9722898d0 2023-11-28T11:21:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c9722898d0 0x55c972289ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d58f9a700 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c9722898d0 0x55c972289ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667f0df624e09c7d server_cookie=fc75ad4fb1b7de39 in_seq=0 out_seq=0 2023-11-28T11:21:29.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.837+0000 7f5d2ffff700 10 client.6240 ms_handle_connect on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d2ffff700 1 -- 172.21.15.53:0/914193252 <== mds.0 v2:172.21.15.53:6833/1970232100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55c97216b210 con 0x55c9722898d0 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d2ffff700 10 client.6240 handle_command_reply: tid=0 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 client.6240 shutdown 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 msgr2=0x55c9723122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5d5002aa10 tx=0x7f5d50015000 comp rx=0 tx=0).stop 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 shutdown_connections 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:6833/1970232100,v1:172.21.15.53:6834/1970232100] conn(0x55c9722898d0 0x55c972289ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c97229c1c0 0x55c972271920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9722991d0 0x55c9723122f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d2ffff700 0 client.6240 ms_handle_reset on v2:172.21.15.53:6833/1970232100 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 --2- 172.21.15.53:0/914193252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c97229b8b0 0x55c97231beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 >> 172.21.15.53:0/914193252 conn(0x55c97226ee00 msgr2=0x55c97228a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 shutdown_connections 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 1 -- 172.21.15.53:0/914193252 wait complete. 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.838+0000 7f5d6f280b80 20 client.6240 trim_cache size 0 max 16384 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-28T11:21:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "backtrace", 2023-11-28T11:21:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 1053960791, 2023-11-28T11:21:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2023-11-28T11:21:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir_test/file" 2023-11-28T11:21:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-28T11:21:29.845 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 msgr2=0x7f5d3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 0x7f5d3806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000f60 tx=0x7f5d50009040 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 msgr2=0x7f5d54151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b680 tx=0x7f5d4c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 shutdown_connections 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5d380682f0 0x7f5d3806a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54161340 0x7f5d54151500 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540d2320 0x7f5d541e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 --2- 172.21.15.53:0/184327341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5415c9b0 0x7f5d5414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 >> 172.21.15.53:0/184327341 conn(0x7f5d54072600 msgr2=0x7f5d54160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 shutdown_connections 2023-11-28T11:21:29.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:29.839+0000 7f5d5b9ff700 1 -- 172.21.15.53:0/184327341 wait complete. 2023-11-28T11:21:29.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.059+0000 7f5f3edff700 1 Processor -- start 2023-11-28T11:21:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.059+0000 7f5f3edff700 1 -- start start 2023-11-28T11:21:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3edff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f3815c9b0 0x7f5f380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f380cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3edff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380d1950 con 0x7f5f38161340 2023-11-28T11:21:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3edff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380b5800 con 0x7f5f3815c9b0 2023-11-28T11:21:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380ce140 con 0x7f5f380cd7c0 2023-11-28T11:21:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3d39c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f380cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f37fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5f38161340 0x7f5f380d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:58148/0 (socket says 172.21.15.53:58148) 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f37fff700 1 -- 172.21.15.53:0/3803370030 learned_addr learned my addr 172.21.15.53:0/3803370030 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.061+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/3803370030 >> v2:172.21.15.157:3300/0 conn(0x7f5f3815c9b0 0x7f5f380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f377fe700 1 -- 172.21.15.53:0/3803370030 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715491039 0 0) 0x7f5f380d1950 con 0x7f5f38161340 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3cb9b700 1 -- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 msgr2=0x7f5f380cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f380cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3cb9b700 1 -- 172.21.15.53:0/3803370030 >> v1:172.21.15.157:6789/0 conn(0x7f5f38161340 legacy=0x7f5f380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3cb9b700 1 -- 172.21.15.53:0/3803370030 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380d24f0 con 0x7f5f3815c9b0 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3d39c700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f380cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380b4bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009840 tx=0x7f5f2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f737a92a71ab7581 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.062+0000 7f5f377fe700 1 -- 172.21.15.53:0/3803370030 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c011830 con 0x7f5f3815c9b0 2023-11-28T11:21:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f377fe700 1 -- 172.21.15.53:0/3803370030 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c011990 con 0x7f5f3815c9b0 2023-11-28T11:21:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f377fe700 1 -- 172.21.15.53:0/3803370030 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c011d20 con 0x7f5f3815c9b0 2023-11-28T11:21:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 -- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 msgr2=0x7f5f380b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380b4bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009840 tx=0x7f5f2c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 -- 172.21.15.53:0/3803370030 shutdown_connections 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f380cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 --2- 172.21.15.53:0/3803370030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380b4bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 -- 172.21.15.53:0/3803370030 >> 172.21.15.53:0/3803370030 conn(0x7f5f38072600 msgr2=0x7f5f380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 -- 172.21.15.53:0/3803370030 shutdown_connections 2023-11-28T11:21:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.063+0000 7f5f3edff700 1 -- 172.21.15.53:0/3803370030 wait complete. 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.065+0000 7f5f3edff700 1 Processor -- start 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.065+0000 7f5f3edff700 1 -- start start 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f3814d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381f33f0 con 0x7f5f38161340 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381f39a0 con 0x7f5f3815c9b0 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3edff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38157ec0 con 0x7f5f380cd7c0 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3d39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3d39c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40132/0 (socket says 172.21.15.53:40132) 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.066+0000 7f5f3d39c700 1 -- 172.21.15.53:0/1456567779 learned_addr learned my addr 172.21.15.53:0/1456567779 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f3814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f37fff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 -- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 msgr2=0x7f5f3814d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f3814d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 -- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 msgr2=0x7f5f380c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 -- 172.21.15.53:0/1456567779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30009a70 con 0x7f5f38161340 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f3814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f37fff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3d39c700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009e60 tx=0x7f5f3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30009cd0 con 0x7f5f38161340 2023-11-28T11:21:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3000f040 con 0x7f5f38161340 2023-11-28T11:21:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000ce20 con 0x7f5f38161340 2023-11-28T11:21:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c0094a0 con 0x7f5f38161340 2023-11-28T11:21:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.067+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380d1300 con 0x7f5f38161340 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.069+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380b5520 con 0x7f5f38161340 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.069+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f30014480 con 0x7f5f38161340 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.071+0000 7f5f357fa700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 0x7f5f1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.072+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f30086590 con 0x7f5f38161340 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.072+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 0x7f5f1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.073+0000 7f5f3cb9b700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 0x7f5f1c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f381f5ed0 tx=0x7f5f2c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.076+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f30052d10 con 0x7f5f38161340 2023-11-28T11:21:30.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.323+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5f380d15e0 con 0x7f5f38161340 2023-11-28T11:21:30.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.323+0000 7f5f357fa700 1 -- 172.21.15.53:0/1456567779 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5f30056330 con 0x7f5f38161340 2023-11-28T11:21:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:30.325 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 msgr2=0x7f5f1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 0x7f5f1c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f381f5ed0 tx=0x7f5f2c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 msgr2=0x7f5f380c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009e60 tx=0x7f5f3000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 shutdown_connections 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f5f1c0682f0 0x7f5f1c06a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380cd7c0 0x7f5f3814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f38161340 0x7f5f380c7ba0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 --2- 172.21.15.53:0/1456567779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815c9b0 0x7f5f380c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:30.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 >> 172.21.15.53:0/1456567779 conn(0x7f5f38072600 msgr2=0x7f5f380b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:30.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 shutdown_connections 2023-11-28T11:21:30.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:30.326+0000 7f5f3edff700 1 -- 172.21.15.53:0/1456567779 wait complete. 2023-11-28T11:21:30.345 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:21:35.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:35.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.559+0000 7f9e739ff700 1 Processor -- start 2023-11-28T11:21:35.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.559+0000 7f9e739ff700 1 -- start start 2023-11-28T11:21:35.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.560+0000 7f9e739ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e6c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e739ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c0ccc00 con 0x7f9e6c159860 2023-11-28T11:21:35.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e739ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c0ccec0 con 0x7f9e6c0cdc80 2023-11-28T11:21:35.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e739ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0cd1a0 con 0x7f9e6c159490 2023-11-28T11:21:35.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e70f9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9e6c159860 0x7f9e6c0c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42294/0 (socket says 172.21.15.53:42294) 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2083287045 learned_addr learned my addr 172.21.15.53:0/2083287045 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e6c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155074510 0 0) 0x7f9e6c0ccc00 con 0x7f9e6c159860 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.561+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e580035b0 con 0x7f9e6c159860 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901961046 0 0) 0x7f9e580035b0 con 0x7f9e6c159860 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 msgr2=0x7f9e6c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e6c0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 >> v1:172.21.15.157:6789/0 conn(0x7f9e6c0cdc80 legacy=0x7f9e6c06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0cd480 con 0x7f9e6c159860 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f9e5c001670 con 0x7f9e6c159860 2023-11-28T11:21:35.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 >> v1:172.21.15.53:6789/0 conn(0x7f9e6c159860 legacy=0x7f9e6c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58003a30 0x7f9e58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e580043d0 0x7f9e58004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e580050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0cd1a0 con 0x7f9e580043d0 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5800ba10 con 0x7f9e58003a30 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5800bba0 con 0x7f9e6c159490 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e580043d0 0x7f9e58004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58003a30 0x7f9e58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e71f9c700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e580050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.562+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 msgr2=0x7f9e580050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:35.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6c159490 0x7f9e580050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58003a30 msgr2=0x7f9e58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58003a30 0x7f9e58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2083287045 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580035b0 con 0x7f9e580043d0 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58003a30 0x7f9e58003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e580043d0 0x7f9e58004890 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c2f0 tx=0x7f9e58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41449b9bf976d6fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00ce30 con 0x7f9e580043d0 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c005960 con 0x7f9e580043d0 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e63fff700 1 -- 172.21.15.53:0/2083287045 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c005d10 con 0x7f9e580043d0 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e739ff700 1 -- 172.21.15.53:0/2083287045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e580043d0 msgr2=0x7f9e58004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.563+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2083287045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e580043d0 0x7f9e58004890 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c2f0 tx=0x7f9e58005880 comp rx=0 tx=0).stop 2023-11-28T11:21:35.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.564+0000 7f9e71f9c700 1 -- 172.21.15.53:0/2083287045 reap_dead start 2023-11-28T11:21:35.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.564+0000 7f9e739ff700 1 -- 172.21.15.53:0/2083287045 shutdown_connections 2023-11-28T11:21:35.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.564+0000 7f9e739ff700 1 -- 172.21.15.53:0/2083287045 >> 172.21.15.53:0/2083287045 conn(0x7f9e6c0cb670 msgr2=0x7f9e6c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:35.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.564+0000 7f9e739ff700 1 -- 172.21.15.53:0/2083287045 shutdown_connections 2023-11-28T11:21:35.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.564+0000 7f9e739ff700 1 -- 172.21.15.53:0/2083287045 wait complete. 2023-11-28T11:21:35.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.566+0000 7f9e739ff700 1 Processor -- start 2023-11-28T11:21:35.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.566+0000 7f9e739ff700 1 -- start start 2023-11-28T11:21:35.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.566+0000 7f9e739ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580043d0 0x7f9e6c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e739ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e739ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6c1f6050 0x7f9e6c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e739ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0adca0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e739ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0b26e0 con 0x7f9e6c1f6050 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e739ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0ad970 con 0x7f9e580043d0 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32916/0 (socket says 172.21.15.53:32916) 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2900720081 learned_addr learned my addr 172.21.15.53:0/2900720081 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580043d0 0x7f9e6c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e71f9c700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6c1f6050 0x7f9e6c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580043d0 msgr2=0x7f9e6c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580043d0 0x7f9e6c0b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6c1f6050 msgr2=0x7f9e6c1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6c1f6050 0x7f9e6c1f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.567+0000 7f9e70f9a700 1 -- 172.21.15.53:0/2900720081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0c9bd0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e70f9a700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001930 tx=0x7f9e5c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c001660 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c005e80 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c004cc0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c0cd760 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.568+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c0c9d60 con 0x7f9e6c153fe0 2023-11-28T11:21:35.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.569+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c0745c0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.570+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e5c024030 con 0x7f9e6c153fe0 2023-11-28T11:21:35.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.571+0000 7f9e61ffb700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 0x7f9e5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:35.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.571+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e5c0870a0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.571+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 0x7f9e5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:35.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.572+0000 7f9e7179b700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 0x7f9e5006a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009d80 tx=0x7f9e68009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:35.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.575+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e5c053870 con 0x7f9e6c153fe0 2023-11-28T11:21:35.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.825+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9e6c0b4b70 con 0x7f9e6c153fe0 2023-11-28T11:21:35.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.826+0000 7f9e61ffb700 1 -- 172.21.15.53:0/2900720081 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9e5c007da0 con 0x7f9e6c153fe0 2023-11-28T11:21:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:35.828 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:21:35.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 msgr2=0x7f9e5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 0x7f9e5006a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009d80 tx=0x7f9e68009210 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 msgr2=0x7f9e6c1f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001930 tx=0x7f9e5c001a00 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 shutdown_connections 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9e500682f0 0x7f9e5006a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580043d0 0x7f9e6c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6c1f6050 0x7f9e6c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 --2- 172.21.15.53:0/2900720081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c153fe0 0x7f9e6c1f5b10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 >> 172.21.15.53:0/2900720081 conn(0x7f9e6c0cb670 msgr2=0x7f9e6c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 shutdown_connections 2023-11-28T11:21:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:35.828+0000 7f9e739ff700 1 -- 172.21.15.53:0/2900720081 wait complete. 2023-11-28T11:21:35.849 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:21:40.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:41.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.060+0000 7fa4eb1ff700 1 Processor -- start 2023-11-28T11:21:41.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.060+0000 7fa4eb1ff700 1 -- start start 2023-11-28T11:21:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4eb1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4e415c9b0 0x7fa4e40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4eb1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4eb1ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40d1950 con 0x7fa4e4161340 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4eb1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40b5800 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4eb1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40ce140 con 0x7fa4e415c9b0 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4e979c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4e979c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39108/0 (socket says 172.21.15.53:39108) 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.061+0000 7fa4e979c700 1 -- 172.21.15.53:0/97116054 learned_addr learned my addr 172.21.15.53:0/97116054 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e8f9b700 1 --2- 172.21.15.53:0/97116054 >> v2:172.21.15.157:3300/0 conn(0x7fa4e415c9b0 0x7fa4e40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e979c700 1 -- 172.21.15.53:0/97116054 >> v2:172.21.15.157:3300/0 conn(0x7fa4e415c9b0 msgr2=0x7fa4e40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e979c700 1 --2- 172.21.15.53:0/97116054 >> v2:172.21.15.157:3300/0 conn(0x7fa4e415c9b0 0x7fa4e40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e979c700 1 -- 172.21.15.53:0/97116054 >> v1:172.21.15.157:6789/0 conn(0x7fa4e4161340 legacy=0x7fa4e40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e979c700 1 -- 172.21.15.53:0/97116054 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40d24f0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e979c700 1 --2- 172.21.15.53:0/97116054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4067460 tx=0x7fa4dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e8ca5aecefd06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e37fe700 1 -- 172.21.15.53:0/97116054 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00e070 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.062+0000 7fa4e37fe700 1 -- 172.21.15.53:0/97116054 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00b9c0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4e37fe700 1 -- 172.21.15.53:0/97116054 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00bd70 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/97116054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 msgr2=0x7fa4e40cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/97116054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4067460 tx=0x7fa4dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/97116054 shutdown_connections 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/97116054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e40cdc00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/97116054 >> v2:172.21.15.157:3300/0 conn(0x7fa4e415c9b0 0x7fa4e40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/97116054 >> 172.21.15.53:0/97116054 conn(0x7fa4e4072600 msgr2=0x7fa4e40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/97116054 shutdown_connections 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.063+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/97116054 wait complete. 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.065+0000 7fa4eb1ff700 1 Processor -- start 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.065+0000 7fa4eb1ff700 1 -- start start 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 0x7fa4e4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 0x7fa4e414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41f51c0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41f45b0 con 0x7fa4e415c9b0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4eb1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41f4890 con 0x7fa4e4161340 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4e8f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4e8f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39116/0 (socket says 172.21.15.53:39116) 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.066+0000 7fa4e8f9b700 1 -- 172.21.15.53:0/3735222072 learned_addr learned my addr 172.21.15.53:0/3735222072 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e3fff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 0x7fa4e4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e979c700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 0x7fa4e414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 -- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 msgr2=0x7fa4e414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 0x7fa4e414a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 -- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 msgr2=0x7fa4e4157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 0x7fa4e4157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 -- 172.21.15.53:0/3735222072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc009b30 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e979c700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 0x7fa4e414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e3fff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 0x7fa4e4157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e8f9b700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d800 tx=0x7fa4d800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80125f0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8012750 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e40d1300 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.067+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8012b00 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.068+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e40b5520 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.068+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40d15e0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.069+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4d8012c60 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.071+0000 7fa4e17fa700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 0x7fa4c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.071+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa4d8086300 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.071+0000 7fa4e3fff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 0x7fa4c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.072+0000 7fa4e3fff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 0x7fa4c806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4154490 tx=0x7fa4d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.075+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4d80529e0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.321+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa4e41f54a0 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.322+0000 7fa4e17fa700 1 -- 172.21.15.53:0/3735222072 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa4d8056000 con 0x7fa4e40cd7c0 2023-11-28T11:21:41.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:41.324 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:21:41.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 msgr2=0x7fa4c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 0x7fa4c806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4154490 tx=0x7fa4d0009040 comp rx=0 tx=0).stop 2023-11-28T11:21:41.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 msgr2=0x7fa4e4154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d800 tx=0x7fa4d800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 shutdown_connections 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4c80682f0 0x7fa4c806a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4e4161340 0x7fa4e414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e40cd7c0 0x7fa4e4154dd0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 --2- 172.21.15.53:0/3735222072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e415c9b0 0x7fa4e4157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 >> 172.21.15.53:0/3735222072 conn(0x7fa4e4072600 msgr2=0x7fa4e4160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 shutdown_connections 2023-11-28T11:21:41.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:41.324+0000 7fa4eb1ff700 1 -- 172.21.15.53:0/3735222072 wait complete. 2023-11-28T11:21:41.345 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:21:45.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:45.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:46.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.582+0000 7fe736bff700 1 Processor -- start 2023-11-28T11:21:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.582+0000 7fe736bff700 1 -- start start 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.583+0000 7fe736bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 0x7fe7300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.583+0000 7fe736bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7300d2320 0x7fe7300cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe736bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7300d1950 con 0x7fe73015c9b0 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe736bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300b5800 con 0x7fe7300d2320 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe736bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300b5300 con 0x7fe730161340 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7300d2320 0x7fe7300cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7300d2320 0x7fe7300cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39128/0 (socket says 172.21.15.53:39128) 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 -- 172.21.15.53:0/519544342 learned_addr learned my addr 172.21.15.53:0/519544342 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73499b700 1 --1- 172.21.15.53:0/519544342 >> v1:172.21.15.53:6789/0 conn(0x7fe73015c9b0 0x7fe7300b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37618/0 (socket says 172.21.15.53:37618) 2023-11-28T11:21:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe72ffff700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 0x7fe7300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 -- 172.21.15.53:0/519544342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 msgr2=0x7fe7300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 0x7fe7300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 -- 172.21.15.53:0/519544342 >> v1:172.21.15.53:6789/0 conn(0x7fe73015c9b0 legacy=0x7fe7300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe73519c700 1 -- 172.21.15.53:0/519544342 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300b5520 con 0x7fe7300d2320 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe72f7fe700 1 -- 172.21.15.53:0/519544342 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702671063 0 0) 0x7fe7300d1950 con 0x7fe73015c9b0 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.584+0000 7fe72ffff700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 0x7fe7300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe73519c700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7300cd7c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe728009e80 tx=0x7fe72800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5182aaab6d1c7248 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe72f7fe700 1 -- 172.21.15.53:0/519544342 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800e070 con 0x7fe7300d2320 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe72f7fe700 1 -- 172.21.15.53:0/519544342 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72800b980 con 0x7fe7300d2320 2023-11-28T11:21:46.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe72f7fe700 1 -- 172.21.15.53:0/519544342 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800bd30 con 0x7fe7300d2320 2023-11-28T11:21:46.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 -- 172.21.15.53:0/519544342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 msgr2=0x7fe7300cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7300cd7c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe728009e80 tx=0x7fe72800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:46.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 -- 172.21.15.53:0/519544342 shutdown_connections 2023-11-28T11:21:46.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7300cd7c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 --2- 172.21.15.53:0/519544342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe730161340 0x7fe7300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.585+0000 7fe736bff700 1 -- 172.21.15.53:0/519544342 >> 172.21.15.53:0/519544342 conn(0x7fe730072600 msgr2=0x7fe7300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.586+0000 7fe736bff700 1 -- 172.21.15.53:0/519544342 shutdown_connections 2023-11-28T11:21:46.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.586+0000 7fe736bff700 1 -- 172.21.15.53:0/519544342 wait complete. 2023-11-28T11:21:46.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.587+0000 7fe736bff700 1 Processor -- start 2023-11-28T11:21:46.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.587+0000 7fe736bff700 1 -- start start 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 0x7fe730150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 0x7fe73015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301567b0 con 0x7fe7300d2320 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730155b30 con 0x7fe730161340 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe736bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730155e10 con 0x7fe73015c9b0 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.588+0000 7fe73499b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39144/0 (socket says 172.21.15.53:39144) 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 -- 172.21.15.53:0/902600038 learned_addr learned my addr 172.21.15.53:0/902600038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:46.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe72ffff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 0x7fe730150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73519c700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 0x7fe73015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 -- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 msgr2=0x7fe730150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 0x7fe730150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 -- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 msgr2=0x7fe73015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 0x7fe73015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 -- 172.21.15.53:0/902600038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009b30 con 0x7fe7300d2320 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe72ffff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 0x7fe730150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73519c700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 0x7fe73015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe73499b700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c550 tx=0x7fe72400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724004c80 con 0x7fe7300d2320 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724004e00 con 0x7fe7300d2320 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.589+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724007310 con 0x7fe7300d2320 2023-11-28T11:21:46.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.590+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7300d15e0 con 0x7fe7300d2320 2023-11-28T11:21:46.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.590+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7300d1300 con 0x7fe7300d2320 2023-11-28T11:21:46.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.591+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73015cd80 con 0x7fe7300d2320 2023-11-28T11:21:46.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.592+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe72401a020 con 0x7fe7300d2320 2023-11-28T11:21:46.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.593+0000 7fe72d7fa700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 0x7fe71406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.593+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe724085b20 con 0x7fe7300d2320 2023-11-28T11:21:46.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.593+0000 7fe72ffff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 0x7fe71406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.594+0000 7fe72ffff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 0x7fe71406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe7300b8a20 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.597+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe724052220 con 0x7fe7300d2320 2023-11-28T11:21:46.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.842+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe7301568f0 con 0x7fe7300d2320 2023-11-28T11:21:46.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.843+0000 7fe72d7fa700 1 -- 172.21.15.53:0/902600038 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe724055840 con 0x7fe7300d2320 2023-11-28T11:21:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:46.845 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:21:46.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 msgr2=0x7fe71406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 0x7fe71406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe7300b8a20 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 msgr2=0x7fe7301538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c550 tx=0x7fe72400c910 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 shutdown_connections 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fe7140682f0 0x7fe71406a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe730161340 0x7fe73015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7300d2320 0x7fe7301538c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 --2- 172.21.15.53:0/902600038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe73015c9b0 0x7fe730150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 >> 172.21.15.53:0/902600038 conn(0x7fe730072600 msgr2=0x7fe73015bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 shutdown_connections 2023-11-28T11:21:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:46.845+0000 7fe736bff700 1 -- 172.21.15.53:0/902600038 wait complete. 2023-11-28T11:21:46.866 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:21:51.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:52.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.080+0000 7fd9dd9ff700 1 Processor -- start 2023-11-28T11:21:52.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.080+0000 7fd9dd9ff700 1 -- start start 2023-11-28T11:21:52.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.081+0000 7fd9dd9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9d80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.081+0000 7fd9dd9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80d1950 con 0x7fd9d8161340 2023-11-28T11:21:52.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9dd9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80ce140 con 0x7fd9d815c9b0 2023-11-28T11:21:52.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9dd9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b5800 con 0x7fd9d80cd7c0 2023-11-28T11:21:52.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9cffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd9d8161340 0x7fd9d80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40546/0 (socket says 172.21.15.53:40546) 2023-11-28T11:21:52.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9cffff700 1 -- 172.21.15.53:0/2977916192 learned_addr learned my addr 172.21.15.53:0/2977916192 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:52.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d77fe700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9d80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585626938 0 0) 0x7fd9d80d1950 con 0x7fd9d8161340 2023-11-28T11:21:52.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c40035b0 con 0x7fd9d8161340 2023-11-28T11:21:52.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759159924 0 0) 0x7fd9d80ce140 con 0x7fd9d815c9b0 2023-11-28T11:21:52.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d80d1950 con 0x7fd9d815c9b0 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243989400 0 0) 0x7fd9c40035b0 con 0x7fd9d8161340 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 msgr2=0x7fd9d80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9d80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 >> v1:172.21.15.157:6789/0 conn(0x7fd9d815c9b0 legacy=0x7fd9d80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.082+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80d24f0 con 0x7fd9d8161340 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fd9c0001670 con 0x7fd9d8161340 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 >> v1:172.21.15.53:6789/0 conn(0x7fd9d8161340 legacy=0x7fd9d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c4003bc0 0x7fd9c4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9c4004850 0x7fd9c4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b5800 con 0x7fd9c4004850 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400bb20 con 0x7fd9c4003bc0 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400bce0 con 0x7fd9d80cd7c0 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9c4004850 0x7fd9c4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d6ffd700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c4003bc0 0x7fd9c4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9d77fe700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 -- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 msgr2=0x7fd9c4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d80cd7c0 0x7fd9c4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 -- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c4003bc0 msgr2=0x7fd9c4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c4003bc0 0x7fd9c4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.083+0000 7fd9cffff700 1 -- 172.21.15.53:0/2977916192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40035b0 con 0x7fd9c4004850 2023-11-28T11:21:52.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9cffff700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9c4004850 0x7fd9c4004cc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c210 tx=0x7fd9c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7583737c3ed7fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000cdb0 con 0x7fd9c4004850 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c0001af0 con 0x7fd9c4004850 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9d67fc700 1 -- 172.21.15.53:0/2977916192 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0005ba0 con 0x7fd9c4004850 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/2977916192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9c4004850 msgr2=0x7fd9c4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/2977916192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9c4004850 0x7fd9c4004cc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c210 tx=0x7fd9c4005880 comp rx=0 tx=0).stop 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9d77fe700 1 -- 172.21.15.53:0/2977916192 reap_dead start 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/2977916192 shutdown_connections 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.084+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/2977916192 >> 172.21.15.53:0/2977916192 conn(0x7fd9d8072600 msgr2=0x7fd9d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.085+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/2977916192 shutdown_connections 2023-11-28T11:21:52.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.085+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/2977916192 wait complete. 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.087+0000 7fd9dd9ff700 1 Processor -- start 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.087+0000 7fd9dd9ff700 1 -- start start 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c4004850 0x7fd9d8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d81f26b0 0x7fd9d81f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81e7600 con 0x7fd9d815e680 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d815f030 con 0x7fd9d81f26b0 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.088+0000 7fd9dd9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81e6a10 con 0x7fd9c4004850 2023-11-28T11:21:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49796/0 (socket says 172.21.15.53:49796) 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 -- 172.21.15.53:0/986018113 learned_addr learned my addr 172.21.15.53:0/986018113 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d6ffd700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c4004850 0x7fd9d8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9cffff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d81f26b0 0x7fd9d81f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 -- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c4004850 msgr2=0x7fd9d8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c4004850 0x7fd9d8157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 -- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d81f26b0 msgr2=0x7fd9d81f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d81f26b0 0x7fd9d81f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 -- 172.21.15.53:0/986018113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80d1300 con 0x7fd9d815e680 2023-11-28T11:21:52.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.089+0000 7fd9d77fe700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd9d808bfd0 tx=0x7fd9d00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.090+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000e070 con 0x7fd9d815e680 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.090+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0003cf0 con 0x7fd9d815e680 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.090+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d80b5520 con 0x7fd9d815e680 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.090+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0003ea0 con 0x7fd9d815e680 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.090+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d80d15e0 con 0x7fd9d815e680 2023-11-28T11:21:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.091+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d80745c0 con 0x7fd9d815e680 2023-11-28T11:21:52.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.092+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd9d0011a40 con 0x7fd9d815e680 2023-11-28T11:21:52.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.093+0000 7fd9d4ff9700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 0x7fd9b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.093+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd9d0086340 con 0x7fd9d815e680 2023-11-28T11:21:52.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.093+0000 7fd9d6ffd700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 0x7fd9b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.094+0000 7fd9d6ffd700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 0x7fd9b806a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80078e0 tx=0x7fd9c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.097+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9d00529c0 con 0x7fd9d815e680 2023-11-28T11:21:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.341+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd9d80d08a0 con 0x7fd9d815e680 2023-11-28T11:21:52.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.342+0000 7fd9d4ff9700 1 -- 172.21.15.53:0/986018113 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd9d0055fe0 con 0x7fd9d815e680 2023-11-28T11:21:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:52.345 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:21:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 msgr2=0x7fd9b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 0x7fd9b806a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80078e0 tx=0x7fd9c8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 msgr2=0x7fd9d815eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd9d808bfd0 tx=0x7fd9d00114b0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 shutdown_connections 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fd9b80682f0 0x7fd9b806a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c4004850 0x7fd9d8157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9d815e680 0x7fd9d815eaf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 --2- 172.21.15.53:0/986018113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d81f26b0 0x7fd9d81f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 >> 172.21.15.53:0/986018113 conn(0x7fd9d8072600 msgr2=0x7fd9d815acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 shutdown_connections 2023-11-28T11:21:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.344+0000 7fd9dd9ff700 1 -- 172.21.15.53:0/986018113 wait complete. 2023-11-28T11:21:52.364 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:21:52.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:21:52.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:21:52.586 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:21:52.586 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:21:52.586 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:52.630 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:21:52.630 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:52.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.958+0000 7f47ffbff700 1 Processor -- start 2023-11-28T11:21:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.959+0000 7f47ffbff700 1 -- start start 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47ffbff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f47f815c9b0 0x7f47f80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47ffbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47ffbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80d1950 con 0x7f47f8161340 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47ffbff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80b5800 con 0x7f47f815c9b0 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47ffbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80ce140 con 0x7f47f80cd7c0 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47fd99b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f47f815c9b0 0x7f47f80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47fd19a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f47f8161340 0x7f47f80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40556/0 (socket says 172.21.15.53:40556) 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47fd19a700 1 -- 172.21.15.53:0/3464693209 learned_addr learned my addr 172.21.15.53:0/3464693209 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.960+0000 7f47fe19c700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fd99b700 1 -- 172.21.15.53:0/3464693209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 msgr2=0x7f47f80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fd99b700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fd99b700 1 -- 172.21.15.53:0/3464693209 >> v1:172.21.15.53:6789/0 conn(0x7f47f8161340 legacy=0x7f47f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fd99b700 1 -- 172.21.15.53:0/3464693209 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80d24f0 con 0x7f47f815c9b0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fc999700 1 -- 172.21.15.53:0/3464693209 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476555188 0 0) 0x7f47f80d1950 con 0x7f47f8161340 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fd99b700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f815c9b0 0x7f47f80b4bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009840 tx=0x7f47f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805eaae40cd26120 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fc999700 1 -- 172.21.15.53:0/3464693209 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4011830 con 0x7f47f815c9b0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fc999700 1 -- 172.21.15.53:0/3464693209 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4011990 con 0x7f47f815c9b0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.961+0000 7f47fc999700 1 -- 172.21.15.53:0/3464693209 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4011d20 con 0x7f47f815c9b0 2023-11-28T11:21:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 -- 172.21.15.53:0/3464693209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f815c9b0 msgr2=0x7f47f80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f815c9b0 0x7f47f80b4bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009840 tx=0x7f47f4009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 -- 172.21.15.53:0/3464693209 shutdown_connections 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 --2- 172.21.15.53:0/3464693209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f815c9b0 0x7f47f80b4bf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 -- 172.21.15.53:0/3464693209 >> 172.21.15.53:0/3464693209 conn(0x7f47f8072600 msgr2=0x7f47f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 -- 172.21.15.53:0/3464693209 shutdown_connections 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.962+0000 7f47ffbff700 1 -- 172.21.15.53:0/3464693209 wait complete. 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.964+0000 7f47ffbff700 1 Processor -- start 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.964+0000 7f47ffbff700 1 -- start start 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f8148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 0x7f47f80cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80aa2c0 con 0x7f47f8161340 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8149620 con 0x7f47f815c9b0 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47ffbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8159a40 con 0x7f47f80cd7c0 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49814/0 (socket says 172.21.15.53:49814) 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 -- 172.21.15.53:0/486561017 learned_addr learned my addr 172.21.15.53:0/486561017 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fd99b700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f8148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fd19a700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 0x7f47f80cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 -- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 msgr2=0x7f47f8148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f8148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 -- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 msgr2=0x7f47f80cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 0x7f47f80cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fe19c700 1 -- 172.21.15.53:0/486561017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40094a0 con 0x7f47f8161340 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.965+0000 7f47fd19a700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 0x7f47f80cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47fe19c700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f47f0007e60 tx=0x7f47f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f000ede0 con 0x7f47f8161340 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f0014420 con 0x7f47f8161340 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80d1300 con 0x7f47f8161340 2023-11-28T11:21:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00147d0 con 0x7f47f8161340 2023-11-28T11:21:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.966+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f80b5520 con 0x7f47f8161340 2023-11-28T11:21:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.967+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f80d15e0 con 0x7f47f8161340 2023-11-28T11:21:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.968+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47f0012050 con 0x7f47f8161340 2023-11-28T11:21:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.969+0000 7f47ee7fc700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 0x7f47dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.969+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f47f0086430 con 0x7f47f8161340 2023-11-28T11:21:52.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.970+0000 7f47fd99b700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 0x7f47dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.971+0000 7f47fd99b700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 0x7f47dc06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f47f40090b0 tx=0x7f47f401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:52.974+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47f0052b10 con 0x7f47f8161340 2023-11-28T11:21:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.176+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47f80745c0 con 0x7f47f8161340 2023-11-28T11:21:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.178+0000 7f47ee7fc700 1 -- 172.21.15.53:0/486561017 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18159 (secure 0 0 0) 0x7f47f0056130 con 0x7f47f8161340 2023-11-28T11:21:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:53.179 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:21:38.857031+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.7","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 msgr2=0x7f47dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 0x7f47dc06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f47f40090b0 tx=0x7f47f401c040 comp rx=0 tx=0).stop 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 msgr2=0x7f47f80cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f47f0007e60 tx=0x7f47f000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 shutdown_connections 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f47dc0682f0 0x7f47dc06a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47f80cd7c0 0x7f47f8148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f8161340 0x7f47f80cb7d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 --2- 172.21.15.53:0/486561017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f815c9b0 0x7f47f80cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 >> 172.21.15.53:0/486561017 conn(0x7f47f8072600 msgr2=0x7f47f815a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 shutdown_connections 2023-11-28T11:21:53.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.180+0000 7f47ffbff700 1 -- 172.21.15.53:0/486561017 wait complete. 2023-11-28T11:21:53.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:46.350181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:47.505564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:21:53.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.404+0000 7f6419fff700 1 Processor -- start 2023-11-28T11:21:53.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.404+0000 7f6419fff700 1 -- start start 2023-11-28T11:21:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f6419fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64140cd7c0 0x7f64140cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f6419fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140d1950 con 0x7f641415c9b0 2023-11-28T11:21:53.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f6419fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140ce140 con 0x7f6414161340 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f6419fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64140b5800 con 0x7f64140cd7c0 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f64137fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f641415c9b0 0x7f64140b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40562/0 (socket says 172.21.15.53:40562) 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f6413fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64140cd7c0 0x7f64140cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.405+0000 7f64137fe700 1 -- 172.21.15.53:0/1627622410 learned_addr learned my addr 172.21.15.53:0/1627622410 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f6413fff700 1 -- 172.21.15.53:0/1627622410 >> v1:172.21.15.157:6789/0 conn(0x7f6414161340 legacy=0x7f64140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f6413fff700 1 -- 172.21.15.53:0/1627622410 >> v1:172.21.15.53:6789/0 conn(0x7f641415c9b0 legacy=0x7f64140b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f6413fff700 1 -- 172.21.15.53:0/1627622410 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140d24f0 con 0x7f64140cd7c0 2023-11-28T11:21:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f6413fff700 1 --2- 172.21.15.53:0/1627622410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64140cd7c0 0x7f64140cdc00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f640c009d90 tx=0x7f640c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f0af582b4f0d65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f64127fc700 1 -- 172.21.15.53:0/1627622410 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00e070 con 0x7f64140cd7c0 2023-11-28T11:21:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f64127fc700 1 -- 172.21.15.53:0/1627622410 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c00bbd0 con 0x7f64140cd7c0 2023-11-28T11:21:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.406+0000 7f64127fc700 1 -- 172.21.15.53:0/1627622410 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c012430 con 0x7f64140cd7c0 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 -- 172.21.15.53:0/1627622410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64140cd7c0 msgr2=0x7f64140cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 --2- 172.21.15.53:0/1627622410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64140cd7c0 0x7f64140cdc00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f640c009d90 tx=0x7f640c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 -- 172.21.15.53:0/1627622410 shutdown_connections 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 --2- 172.21.15.53:0/1627622410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64140cd7c0 0x7f64140cdc00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 -- 172.21.15.53:0/1627622410 >> 172.21.15.53:0/1627622410 conn(0x7f6414072600 msgr2=0x7f64140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 -- 172.21.15.53:0/1627622410 shutdown_connections 2023-11-28T11:21:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.407+0000 7f6419fff700 1 -- 172.21.15.53:0/1627622410 wait complete. 2023-11-28T11:21:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.409+0000 7f6419fff700 1 Processor -- start 2023-11-28T11:21:53.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.409+0000 7f6419fff700 1 -- start start 2023-11-28T11:21:53.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 0x7f6414156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 0x7f6414154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141f2930 con 0x7f641415c9b0 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414157040 con 0x7f6414161340 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6419fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64141567b0 con 0x7f64140cd7c0 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49830/0 (socket says 172.21.15.53:49830) 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6412ffd700 1 -- 172.21.15.53:0/4265492092 learned_addr learned my addr 172.21.15.53:0/4265492092 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f64137fe700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 0x7f6414156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.410+0000 7f6413fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 0x7f6414154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 -- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 msgr2=0x7f6414156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 0x7f6414156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 -- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 msgr2=0x7f6414154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 0x7f6414154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 -- 172.21.15.53:0/4265492092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c009a70 con 0x7f641415c9b0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6413fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 0x7f6414154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f64137fe700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 0x7f6414156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6412ffd700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f640800c4e0 tx=0x7f640800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080052d0 con 0x7f641415c9b0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6408005430 con 0x7f641415c9b0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080038c0 con 0x7f641415c9b0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64140d1300 con 0x7f641415c9b0 2023-11-28T11:21:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.411+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64140b5520 con 0x7f641415c9b0 2023-11-28T11:21:53.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.413+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64140d15e0 con 0x7f641415c9b0 2023-11-28T11:21:53.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.413+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6408019030 con 0x7f641415c9b0 2023-11-28T11:21:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.414+0000 7f63fbfff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.414+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f64080850f0 con 0x7f641415c9b0 2023-11-28T11:21:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.415+0000 7f64137fe700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.415+0000 7f64137fe700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 0x7f640006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6404007fd0 tx=0x7f6404009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.418+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6408051860 con 0x7f641415c9b0 2023-11-28T11:21:53.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.619+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64141f2a70 con 0x7f641415c9b0 2023-11-28T11:21:53.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.619+0000 7f63fbfff700 1 -- 172.21.15.53:0/4265492092 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6408054e80 con 0x7f641415c9b0 2023-11-28T11:21:53.620 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:21:53.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 msgr2=0x7f640006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 0x7f640006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6404007fd0 tx=0x7f6404009040 comp rx=0 tx=0).stop 2023-11-28T11:21:53.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 msgr2=0x7f64141596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f640800c4e0 tx=0x7f640800c8a0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 shutdown_connections 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64140cd7c0 0x7f6414156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6414161340 0x7f6414154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 --2- 172.21.15.53:0/4265492092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641415c9b0 0x7f64141596a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 >> 172.21.15.53:0/4265492092 conn(0x7f6414072600 msgr2=0x7f641406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 shutdown_connections 2023-11-28T11:21:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.622+0000 7f6419fff700 1 -- 172.21.15.53:0/4265492092 wait complete. 2023-11-28T11:21:53.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.844+0000 7f32d63ff700 1 Processor -- start 2023-11-28T11:21:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.844+0000 7f32d63ff700 1 -- start start 2023-11-28T11:21:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.846+0000 7f32d63ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32d0158ce0 0x7f32d015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.846+0000 7f32d63ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 0x7f32d00cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.846+0000 7f32d63ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d006bf10 con 0x7f32d00ce2e0 2023-11-28T11:21:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.846+0000 7f32d63ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00ce720 con 0x7f32d0158ce0 2023-11-28T11:21:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.846+0000 7f32d63ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015d730 con 0x7f32d006fb30 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32d499c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32d00ce2e0 0x7f32d006c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40568/0 (socket says 172.21.15.53:40568) 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32d499c700 1 -- 172.21.15.53:0/4269787609 learned_addr learned my addr 172.21.15.53:0/4269787609 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cffff700 1 --2- 172.21.15.53:0/4269787609 >> v2:172.21.15.157:3300/0 conn(0x7f32d0158ce0 0x7f32d015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cf7fe700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 0x7f32d00cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32ceffd700 1 -- 172.21.15.53:0/4269787609 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284284985 0 0) 0x7f32d006bf10 con 0x7f32d00ce2e0 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32ceffd700 1 -- 172.21.15.53:0/4269787609 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b80035b0 con 0x7f32d00ce2e0 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cffff700 1 -- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 msgr2=0x7f32d00cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cffff700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 0x7f32d00cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cffff700 1 -- 172.21.15.53:0/4269787609 >> v1:172.21.15.53:6789/0 conn(0x7f32d00ce2e0 legacy=0x7f32d006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cffff700 1 -- 172.21.15.53:0/4269787609 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d01590b0 con 0x7f32d0158ce0 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.847+0000 7f32cf7fe700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 0x7f32d00cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32cffff700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d015d1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009840 tx=0x7f32c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460b2a1bf5c7ce8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32ceffd700 1 -- 172.21.15.53:0/4269787609 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0011830 con 0x7f32d0158ce0 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32ceffd700 1 -- 172.21.15.53:0/4269787609 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0011990 con 0x7f32d0158ce0 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32ceffd700 1 -- 172.21.15.53:0/4269787609 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0011d00 con 0x7f32d0158ce0 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 -- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 msgr2=0x7f32d015d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d015d1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009840 tx=0x7f32c0009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 -- 172.21.15.53:0/4269787609 shutdown_connections 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d015d1f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 --2- 172.21.15.53:0/4269787609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d006fb30 0x7f32d00cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.848+0000 7f32d63ff700 1 -- 172.21.15.53:0/4269787609 >> 172.21.15.53:0/4269787609 conn(0x7f32d00cb670 msgr2=0x7f32d00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.849+0000 7f32d63ff700 1 -- 172.21.15.53:0/4269787609 shutdown_connections 2023-11-28T11:21:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.849+0000 7f32d63ff700 1 -- 172.21.15.53:0/4269787609 wait complete. 2023-11-28T11:21:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.851+0000 7f32d63ff700 1 Processor -- start 2023-11-28T11:21:53.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.851+0000 7f32d63ff700 1 -- start start 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d00ce2e0 0x7f32d00c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c3140 con 0x7f32d006fb30 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c3420 con 0x7f32d0158ce0 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.852+0000 7f32d63ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c3700 con 0x7f32d00ce2e0 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49836/0 (socket says 172.21.15.53:49836) 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 -- 172.21.15.53:0/2161262805 learned_addr learned my addr 172.21.15.53:0/2161262805 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cf7fe700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d00ce2e0 0x7f32d00c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32d499c700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 -- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d00ce2e0 msgr2=0x7f32d00c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d00ce2e0 0x7f32d00c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 -- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 msgr2=0x7f32d01f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d01f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.853+0000 7f32cffff700 1 -- 172.21.15.53:0/2161262805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c400a640 con 0x7f32d006fb30 2023-11-28T11:21:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.854+0000 7f32cffff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32c00090b0 tx=0x7f32c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.855+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004550 con 0x7f32d006fb30 2023-11-28T11:21:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.855+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c001e070 con 0x7f32d006fb30 2023-11-28T11:21:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.855+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c00094a0 con 0x7f32d006fb30 2023-11-28T11:21:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.855+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00038e0 con 0x7f32d006fb30 2023-11-28T11:21:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.855+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d00cb3b0 con 0x7f32d006fb30 2023-11-28T11:21:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.856+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d00cb0d0 con 0x7f32d006fb30 2023-11-28T11:21:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.857+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32c000f050 con 0x7f32d006fb30 2023-11-28T11:21:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.858+0000 7f32ccff9700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 0x7f32bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.858+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f32c0086b50 con 0x7f32d006fb30 2023-11-28T11:21:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.859+0000 7f32cf7fe700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 0x7f32bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.859+0000 7f32cf7fe700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 0x7f32bc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a610 tx=0x7f32c400a320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:53.864+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32c0053320 con 0x7f32d006fb30 2023-11-28T11:21:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.067+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f32d006b8c0 con 0x7f32d006fb30 2023-11-28T11:21:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.067+0000 7f32ccff9700 1 -- 172.21.15.53:0/2161262805 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v63) v1 ==== 92+0+10 (secure 0 0 0) 0x7f32c00150a0 con 0x7f32d006fb30 2023-11-28T11:21:54.069 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:21:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 msgr2=0x7f32bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 0x7f32bc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a610 tx=0x7f32c400a320 comp rx=0 tx=0).stop 2023-11-28T11:21:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 msgr2=0x7f32d01f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32c00090b0 tx=0x7f32c0004750 comp rx=0 tx=0).stop 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 shutdown_connections 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f32bc0682f0 0x7f32bc06a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32d0158ce0 0x7f32d01f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d006fb30 0x7f32d01f2460 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 --2- 172.21.15.53:0/2161262805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32d00ce2e0 0x7f32d00c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 >> 172.21.15.53:0/2161262805 conn(0x7f32d00cb670 msgr2=0x7f32d0157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 shutdown_connections 2023-11-28T11:21:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.070+0000 7f32d63ff700 1 -- 172.21.15.53:0/2161262805 wait complete. 2023-11-28T11:21:54.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:54.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.292+0000 7f65157ff700 1 Processor -- start 2023-11-28T11:21:54.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.292+0000 7f65157ff700 1 -- start start 2023-11-28T11:21:54.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.293+0000 7f65157ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f651015c9b0 0x7f65100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.293+0000 7f65157ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6510161340 0x7f65100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.293+0000 7f65157ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65100d1950 con 0x7f65100d2320 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.293+0000 7f65157ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100b5800 con 0x7f6510161340 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.293+0000 7f65157ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100b5520 con 0x7f651015c9b0 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6510161340 0x7f65100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6510161340 0x7f65100d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49840/0 (socket says 172.21.15.53:49840) 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 -- 172.21.15.53:0/37047457 learned_addr learned my addr 172.21.15.53:0/37047457 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650effd700 1 --2- 172.21.15.53:0/37047457 >> v2:172.21.15.157:3300/0 conn(0x7f651015c9b0 0x7f65100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 -- 172.21.15.53:0/37047457 >> v2:172.21.15.157:3300/0 conn(0x7f651015c9b0 msgr2=0x7f65100b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 --2- 172.21.15.53:0/37047457 >> v2:172.21.15.157:3300/0 conn(0x7f651015c9b0 0x7f65100b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 -- 172.21.15.53:0/37047457 >> v1:172.21.15.157:6789/0 conn(0x7f65100d2320 legacy=0x7f65100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.294+0000 7f650e7fc700 1 -- 172.21.15.53:0/37047457 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100d1300 con 0x7f6510161340 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f650e7fc700 1 --2- 172.21.15.53:0/37047457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6510161340 0x7f65100d1de0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6510067460 tx=0x7f6504009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fc401badbaf281 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f650dffb700 1 -- 172.21.15.53:0/37047457 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011830 con 0x7f6510161340 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f650dffb700 1 -- 172.21.15.53:0/37047457 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504011990 con 0x7f6510161340 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f650dffb700 1 -- 172.21.15.53:0/37047457 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011d20 con 0x7f6510161340 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 -- 172.21.15.53:0/37047457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6510161340 msgr2=0x7f65100d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 --2- 172.21.15.53:0/37047457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6510161340 0x7f65100d1de0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6510067460 tx=0x7f6504009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 -- 172.21.15.53:0/37047457 shutdown_connections 2023-11-28T11:21:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 --2- 172.21.15.53:0/37047457 >> v2:172.21.15.157:3300/0 conn(0x7f651015c9b0 0x7f65100b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 --2- 172.21.15.53:0/37047457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6510161340 0x7f65100d1de0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.295+0000 7f65157ff700 1 -- 172.21.15.53:0/37047457 >> 172.21.15.53:0/37047457 conn(0x7f6510072600 msgr2=0x7f65100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.296+0000 7f65157ff700 1 -- 172.21.15.53:0/37047457 shutdown_connections 2023-11-28T11:21:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.296+0000 7f65157ff700 1 -- 172.21.15.53:0/37047457 wait complete. 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.298+0000 7f65157ff700 1 Processor -- start 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.298+0000 7f65157ff700 1 -- start start 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f651015c9b0 0x7f65100c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 0x7f6510159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c8c80 con 0x7f65100d2320 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c8f60 con 0x7f6510161340 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.299+0000 7f65157ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c9240 con 0x7f651015c9b0 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49852/0 (socket says 172.21.15.53:49852) 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 -- 172.21.15.53:0/4139030605 learned_addr learned my addr 172.21.15.53:0/4139030605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 -- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f651015c9b0 msgr2=0x7f65100c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f651015c9b0 0x7f65100c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 -- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 msgr2=0x7f6510159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650f7fe700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 0x7f6510159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 0x7f6510159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 -- 172.21.15.53:0/4139030605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504009550 con 0x7f65100d2320 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650f7fe700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 0x7f6510159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.300+0000 7f650effd700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9b0 tx=0x7f650000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.301+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500009710 con 0x7f65100d2320 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.301+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650000c850 con 0x7f65100d2320 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.301+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65100b5130 con 0x7f65100d2320 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.301+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650000cbc0 con 0x7f65100d2320 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.301+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65100d15e0 con 0x7f65100d2320 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.302+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65100b52c0 con 0x7f65100d2320 2023-11-28T11:21:54.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.303+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6500019030 con 0x7f65100d2320 2023-11-28T11:21:54.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.304+0000 7f64f77fe700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 0x7f64fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.304+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6500085d60 con 0x7f65100d2320 2023-11-28T11:21:54.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.304+0000 7f650e7fc700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 0x7f64fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.305+0000 7f650e7fc700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 0x7f64fc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65100ce3a0 tx=0x7f6504010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.308+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6500052530 con 0x7f65100d2320 2023-11-28T11:21:54.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.514+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65100745c0 con 0x7f65100d2320 2023-11-28T11:21:54.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.515+0000 7f64f77fe700 1 -- 172.21.15.53:0/4139030605 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6500017020 con 0x7f65100d2320 2023-11-28T11:21:54.516 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:21:54.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 msgr2=0x7f64fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 0x7f64fc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65100ce3a0 tx=0x7f6504010000 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 msgr2=0x7f65101468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9b0 tx=0x7f650000ed70 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 shutdown_connections 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64fc0682f0 0x7f64fc06a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510161340 0x7f6510159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65100d2320 0x7f65101468d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 --2- 172.21.15.53:0/4139030605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f651015c9b0 0x7f65100c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 >> 172.21.15.53:0/4139030605 conn(0x7f6510072600 msgr2=0x7f651015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.517+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 shutdown_connections 2023-11-28T11:21:54.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.518+0000 7f65157ff700 1 -- 172.21.15.53:0/4139030605 wait complete. 2023-11-28T11:21:54.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:54.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.741+0000 7fb45a7ff700 1 Processor -- start 2023-11-28T11:21:54.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.741+0000 7fb45a7ff700 1 -- start start 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb45a7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb454161340 0x7fb4540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb45a7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4540d1950 con 0x7fb4540d2320 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb45a7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4540b5520 con 0x7fb45415c9b0 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb45a7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4540b5800 con 0x7fb454161340 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb458d9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb4540d2320 0x7fb4540cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40582/0 (socket says 172.21.15.53:40582) 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb458d9c700 1 -- 172.21.15.53:0/595936101 learned_addr learned my addr 172.21.15.53:0/595936101 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:54.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb4537fe700 1 --2- 172.21.15.53:0/595936101 >> v2:172.21.15.157:3300/0 conn(0x7fb454161340 0x7fb4540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.742+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610571597 0 0) 0x7fb4540d1950 con 0x7fb4540d2320 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb43c0035b0 con 0x7fb4540d2320 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091369381 0 0) 0x7fb4540b5520 con 0x7fb45415c9b0 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4540d1950 con 0x7fb45415c9b0 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180216160 0 0) 0x7fb43c0035b0 con 0x7fb4540d2320 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb4537fe700 1 -- 172.21.15.53:0/595936101 >> v1:172.21.15.157:6789/0 conn(0x7fb45415c9b0 legacy=0x7fb4540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb4537fe700 1 -- 172.21.15.53:0/595936101 >> v1:172.21.15.53:6789/0 conn(0x7fb4540d2320 legacy=0x7fb4540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb4537fe700 1 -- 172.21.15.53:0/595936101 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540d1300 con 0x7fb454161340 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb4537fe700 1 --2- 172.21.15.53:0/595936101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb454161340 0x7fb4540d1de0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb448009840 tx=0x7fb448009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fad17bbbb30c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0019f0 con 0x7fb454161340 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448004030 con 0x7fb454161340 2023-11-28T11:21:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.743+0000 7fb452ffd700 1 -- 172.21.15.53:0/595936101 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480043c0 con 0x7fb454161340 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 -- 172.21.15.53:0/595936101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb454161340 msgr2=0x7fb4540d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/595936101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb454161340 0x7fb4540d1de0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb448009840 tx=0x7fb448009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 -- 172.21.15.53:0/595936101 shutdown_connections 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/595936101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb454161340 0x7fb4540d1de0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 -- 172.21.15.53:0/595936101 >> 172.21.15.53:0/595936101 conn(0x7fb454072600 msgr2=0x7fb4540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 -- 172.21.15.53:0/595936101 shutdown_connections 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.744+0000 7fb45a7ff700 1 -- 172.21.15.53:0/595936101 wait complete. 2023-11-28T11:21:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.746+0000 7fb45a7ff700 1 Processor -- start 2023-11-28T11:21:54.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.747+0000 7fb45a7ff700 1 -- start start 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 0x7fb4540b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb454161340 0x7fb454156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541e7a80 con 0x7fb45415c9b0 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541e6e70 con 0x7fb4540d2320 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb45a7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541e7150 con 0x7fb454161340 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49856/0 (socket says 172.21.15.53:49856) 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb4537fe700 1 -- 172.21.15.53:0/1725092868 learned_addr learned my addr 172.21.15.53:0/1725092868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:54.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb453fff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 0x7fb4540b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.748+0000 7fb458d9c700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb454161340 0x7fb454156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 -- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb454161340 msgr2=0x7fb454156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb454161340 0x7fb454156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 -- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 msgr2=0x7fb4540b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 0x7fb4540b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 -- 172.21.15.53:0/1725092868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480094a0 con 0x7fb45415c9b0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb453fff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 0x7fb4540b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb4537fe700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb44800a010 tx=0x7fb448004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003db0 con 0x7fb45415c9b0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44801f070 con 0x7fb45415c9b0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45415cd80 con 0x7fb45415c9b0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448011870 con 0x7fb45415c9b0 2023-11-28T11:21:54.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.749+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4540d15e0 con 0x7fb45415c9b0 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.750+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45415cf10 con 0x7fb45415c9b0 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.751+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb44802a020 con 0x7fb45415c9b0 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.752+0000 7fb450ff9700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 0x7fb44006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.752+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb448087320 con 0x7fb45415c9b0 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.753+0000 7fb453fff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 0x7fb44006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.753+0000 7fb453fff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 0x7fb44006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb4541e9440 tx=0x7fb444009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.757+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb448053a00 con 0x7fb45415c9b0 2023-11-28T11:21:54.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.960+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb454160070 con 0x7fb45415c9b0 2023-11-28T11:21:54.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.960+0000 7fb450ff9700 1 -- 172.21.15.53:0/1725092868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb448057020 con 0x7fb45415c9b0 2023-11-28T11:21:54.961 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 msgr2=0x7fb44006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 0x7fb44006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb4541e9440 tx=0x7fb444009040 comp rx=0 tx=0).stop 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 msgr2=0x7fb45406ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb44800a010 tx=0x7fb448004750 comp rx=0 tx=0).stop 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 shutdown_connections 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb4400682f0 0x7fb44006a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb454161340 0x7fb454156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4540d2320 0x7fb4540b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 --2- 172.21.15.53:0/1725092868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45415c9b0 0x7fb45406ab70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 >> 172.21.15.53:0/1725092868 conn(0x7fb454072600 msgr2=0x7fb4540b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 shutdown_connections 2023-11-28T11:21:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:54.963+0000 7fb45a7ff700 1 -- 172.21.15.53:0/1725092868 wait complete. 2023-11-28T11:21:54.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:21:55.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.184+0000 7f58493ff700 1 Processor -- start 2023-11-28T11:21:55.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.184+0000 7f58493ff700 1 -- start start 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f58493ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5844161340 0x7f58440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f58493ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58440d2320 0x7f58440cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f58493ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440d1950 con 0x7f584415c9b0 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f58493ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58440b5800 con 0x7f5844161340 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f58493ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58440b5300 con 0x7f58440d2320 2023-11-28T11:21:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f583bfff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5844161340 0x7f58440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f5842ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f584415c9b0 0x7f58440b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40586/0 (socket says 172.21.15.53:40586) 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.185+0000 7f5842ffd700 1 -- 172.21.15.53:0/2867383332 learned_addr learned my addr 172.21.15.53:0/2867383332 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f58437fe700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58440d2320 0x7f58440cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f583bfff700 1 -- 172.21.15.53:0/2867383332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58440d2320 msgr2=0x7f58440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f583bfff700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58440d2320 0x7f58440cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f583bfff700 1 -- 172.21.15.53:0/2867383332 >> v1:172.21.15.53:6789/0 conn(0x7f584415c9b0 legacy=0x7f58440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f583bfff700 1 -- 172.21.15.53:0/2867383332 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440b5520 con 0x7f5844161340 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f58427fc700 1 -- 172.21.15.53:0/2867383332 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016834297 0 0) 0x7f58440d1950 con 0x7f584415c9b0 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f583bfff700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5844161340 0x7f58440d1de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f582c009840 tx=0x7f582c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dea39aa014a86d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f58427fc700 1 -- 172.21.15.53:0/2867383332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c011830 con 0x7f5844161340 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f58427fc700 1 -- 172.21.15.53:0/2867383332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c011990 con 0x7f5844161340 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.186+0000 7f58427fc700 1 -- 172.21.15.53:0/2867383332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c011d20 con 0x7f5844161340 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 -- 172.21.15.53:0/2867383332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5844161340 msgr2=0x7f58440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5844161340 0x7f58440d1de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f582c009840 tx=0x7f582c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 -- 172.21.15.53:0/2867383332 shutdown_connections 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58440d2320 0x7f58440cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 --2- 172.21.15.53:0/2867383332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5844161340 0x7f58440d1de0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 -- 172.21.15.53:0/2867383332 >> 172.21.15.53:0/2867383332 conn(0x7f5844072600 msgr2=0x7f58440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 -- 172.21.15.53:0/2867383332 shutdown_connections 2023-11-28T11:21:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.187+0000 7f58493ff700 1 -- 172.21.15.53:0/2867383332 wait complete. 2023-11-28T11:21:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.189+0000 7f58493ff700 1 Processor -- start 2023-11-28T11:21:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.189+0000 7f58493ff700 1 -- start start 2023-11-28T11:21:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58440d2320 0x7f58440c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5844161340 0x7f58441516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440c7f40 con 0x7f584415c9b0 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440ce1b0 con 0x7f58440d2320 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58493ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58440c7330 con 0x7f5844161340 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49862/0 (socket says 172.21.15.53:49862) 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 -- 172.21.15.53:0/3981549412 learned_addr learned my addr 172.21.15.53:0/3981549412 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f5842ffd700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58440d2320 0x7f58440c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f58437fe700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5844161340 0x7f58441516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 -- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5844161340 msgr2=0x7f58441516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5844161340 0x7f58441516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 -- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58440d2320 msgr2=0x7f58440c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58440d2320 0x7f58440c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.190+0000 7f583bfff700 1 -- 172.21.15.53:0/3981549412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0094a0 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f583bfff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f582c009b90 tx=0x7f582c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c004550 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c01f070 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440d15e0 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0038e0 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.191+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58440d1300 con 0x7f584415c9b0 2023-11-28T11:21:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.192+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584415cd80 con 0x7f584415c9b0 2023-11-28T11:21:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.193+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f582c00f050 con 0x7f584415c9b0 2023-11-28T11:21:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.194+0000 7f583b7fe700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 0x7f582406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.194+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f582c021070 con 0x7f584415c9b0 2023-11-28T11:21:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.195+0000 7f5842ffd700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 0x7f582406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.195+0000 7f5842ffd700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 0x7f582406a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f5834007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.198+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f582c053d80 con 0x7f584415c9b0 2023-11-28T11:21:55.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.418+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58440745c0 con 0x7f584415c9b0 2023-11-28T11:21:55.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.418+0000 7f583b7fe700 1 -- 172.21.15.53:0/3981549412 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7f582c015080 con 0x7f584415c9b0 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 msgr2=0x7f582406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 0x7f582406a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f5834007e70 comp rx=0 tx=0).stop 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 msgr2=0x7f58440cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f582c009b90 tx=0x7f582c004750 comp rx=0 tx=0).stop 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 shutdown_connections 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f58240682f0 0x7f582406a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5844161340 0x7f58441516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58440d2320 0x7f58440c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 --2- 172.21.15.53:0/3981549412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584415c9b0 0x7f58440cdc70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 >> 172.21.15.53:0/3981549412 conn(0x7f5844072600 msgr2=0x7f58440b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 shutdown_connections 2023-11-28T11:21:55.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.421+0000 7f58493ff700 1 -- 172.21.15.53:0/3981549412 wait complete. 2023-11-28T11:21:55.424 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-11-28T11:21:55.440 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:21:55.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:55.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:55.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:21:55.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:55.540 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:21:55.593 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:21:55.593 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:21:55.640 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:21:55.640 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:21:55.684 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:21:55.685 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:21:55.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:55.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.930+0000 7f73079ff700 1 Processor -- start 2023-11-28T11:21:55.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.930+0000 7f73079ff700 1 -- start start 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f730015c9b0 0x7f73000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7300161340 0x7f73000d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 0x7f73000cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000b5800 con 0x7f730015c9b0 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000d1950 con 0x7f7300161340 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f73079ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000b5520 con 0x7f73000d2320 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f730579b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f730015c9b0 0x7f73000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f730579b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f730015c9b0 0x7f73000b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49876/0 (socket says 172.21.15.53:49876) 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.931+0000 7f730579b700 1 -- 172.21.15.53:0/2105733249 learned_addr learned my addr 172.21.15.53:0/2105733249 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f7305f9c700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 0x7f73000cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f7304f9a700 1 --2- 172.21.15.53:0/2105733249 >> v2:172.21.15.157:3300/0 conn(0x7f7300161340 0x7f73000d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 -- 172.21.15.53:0/2105733249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 msgr2=0x7f73000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 0x7f73000cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 -- 172.21.15.53:0/2105733249 >> v2:172.21.15.157:3300/0 conn(0x7f7300161340 msgr2=0x7f73000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 --2- 172.21.15.53:0/2105733249 >> v2:172.21.15.157:3300/0 conn(0x7f7300161340 0x7f73000d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 -- 172.21.15.53:0/2105733249 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000d1300 con 0x7f730015c9b0 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f7305f9c700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 0x7f73000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f730579b700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f730015c9b0 0x7f73000b4bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f73000685c0 tx=0x7f72fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5b99c0c8289281 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f72f7fff700 1 -- 172.21.15.53:0/2105733249 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00b870 con 0x7f730015c9b0 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.932+0000 7f72f7fff700 1 -- 172.21.15.53:0/2105733249 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00b9d0 con 0x7f730015c9b0 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f72f7fff700 1 -- 172.21.15.53:0/2105733249 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00bd60 con 0x7f730015c9b0 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 -- 172.21.15.53:0/2105733249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f730015c9b0 msgr2=0x7f73000b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f730015c9b0 0x7f73000b4bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f73000685c0 tx=0x7f72fc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 -- 172.21.15.53:0/2105733249 shutdown_connections 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73000d2320 0x7f73000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 --2- 172.21.15.53:0/2105733249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f730015c9b0 0x7f73000b4bf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 --2- 172.21.15.53:0/2105733249 >> v2:172.21.15.157:3300/0 conn(0x7f7300161340 0x7f73000d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 -- 172.21.15.53:0/2105733249 >> 172.21.15.53:0/2105733249 conn(0x7f7300072600 msgr2=0x7f73000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 -- 172.21.15.53:0/2105733249 shutdown_connections 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.933+0000 7f73079ff700 1 -- 172.21.15.53:0/2105733249 wait complete. 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.935+0000 7f73079ff700 1 Processor -- start 2023-11-28T11:21:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.935+0000 7f73079ff700 1 -- start start 2023-11-28T11:21:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730015c9b0 0x7f7300155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 0x7f730015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300157fb0 con 0x7f73000d2320 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300155020 con 0x7f730015c9b0 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f73079ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73001f5870 con 0x7f7300161340 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f730579b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f730579b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49892/0 (socket says 172.21.15.53:49892) 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.936+0000 7f730579b700 1 -- 172.21.15.53:0/3154672969 learned_addr learned my addr 172.21.15.53:0/3154672969 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f7304f9a700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730015c9b0 0x7f7300155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f7305f9c700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 0x7f730015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 -- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 msgr2=0x7f730015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 0x7f730015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 -- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730015c9b0 msgr2=0x7f7300155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730015c9b0 0x7f7300155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 -- 172.21.15.53:0/3154672969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc009550 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f7305f9c700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 0x7f730015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f730579b700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f72fc000c00 tx=0x7f72fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc004540 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc0036a0 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73000b5130 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc003a50 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.937+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73000d15e0 con 0x7f73000d2320 2023-11-28T11:21:55.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.938+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73000b52c0 con 0x7f73000d2320 2023-11-28T11:21:55.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.939+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f72fc00f050 con 0x7f73000d2320 2023-11-28T11:21:55.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.940+0000 7f72f5ffb700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 0x7f72e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.940+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f72fc0876f0 con 0x7f73000d2320 2023-11-28T11:21:55.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.941+0000 7f7304f9a700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 0x7f72e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.941+0000 7f7304f9a700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 0x7f72e406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f72f00098b0 tx=0x7f72f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:55.944+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f72fc053dd0 con 0x7f73000d2320 2023-11-28T11:21:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.192+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73000745c0 con 0x7f73000d2320 2023-11-28T11:21:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.192+0000 7f72f5ffb700 1 -- 172.21.15.53:0/3154672969 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f72fc0573f0 con 0x7f73000d2320 2023-11-28T11:21:56.193 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:56.194 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:21:04.863310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:21:56.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 msgr2=0x7f72e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 0x7f72e406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f72f00098b0 tx=0x7f72f0008040 comp rx=0 tx=0).stop 2023-11-28T11:21:56.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 msgr2=0x7f7300157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f72fc000c00 tx=0x7f72fc004750 comp rx=0 tx=0).stop 2023-11-28T11:21:56.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 shutdown_connections 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f72e40682f0 0x7f72e406a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7300161340 0x7f730015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73000d2320 0x7f7300157a70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 --2- 172.21.15.53:0/3154672969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730015c9b0 0x7f7300155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 >> 172.21.15.53:0/3154672969 conn(0x7f7300072600 msgr2=0x7f730015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 shutdown_connections 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.195+0000 7f73079ff700 1 -- 172.21.15.53:0/3154672969 wait complete. 2023-11-28T11:21:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 33 2023-11-28T11:21:56.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.415+0000 7fecf19ff700 1 Processor -- start 2023-11-28T11:21:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.415+0000 7fecf19ff700 1 -- start start 2023-11-28T11:21:56.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.416+0000 7fecf19ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecec161340 0x7fecec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.416+0000 7fecf19ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecec0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecf19ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0d1950 con 0x7fecec15c9b0 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecf19ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec0b5800 con 0x7fecec161340 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecf19ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b5300 con 0x7fecec0d2320 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7feceaffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fecec15c9b0 0x7fecec0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40602/0 (socket says 172.21.15.53:40602) 2023-11-28T11:21:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7feceaffd700 1 -- 172.21.15.53:0/819504591 learned_addr learned my addr 172.21.15.53:0/819504591 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fece3fff700 1 --2- 172.21.15.53:0/819504591 >> v2:172.21.15.157:3300/0 conn(0x7fecec161340 0x7fecec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7feceb7fe700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecec0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253888106 0 0) 0x7fecec0d1950 con 0x7fecec15c9b0 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd80035b0 con 0x7fecec15c9b0 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.417+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738619604 0 0) 0x7fecd80035b0 con 0x7fecec15c9b0 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 msgr2=0x7fecec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecec0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 >> v2:172.21.15.157:3300/0 conn(0x7fecec161340 msgr2=0x7fecec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 --2- 172.21.15.53:0/819504591 >> v2:172.21.15.157:3300/0 conn(0x7fecec161340 0x7fecec0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0b5520 con 0x7fecec15c9b0 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fecdc001670 con 0x7fecec15c9b0 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 >> v1:172.21.15.53:6789/0 conn(0x7fecec15c9b0 legacy=0x7fecec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd8003a00 0x7fecd8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecd8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d90 0x7fecd8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b5800 con 0x7fecd8003a00 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b5300 con 0x7fecd8004d90 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd800ba60 con 0x7fecec0d2320 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceaffd700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd8003a00 0x7fecd8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7fece3fff700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecd8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceb7fe700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d90 0x7fecd8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceaffd700 1 -- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 msgr2=0x7fecd8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceaffd700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0d2320 0x7fecd8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceaffd700 1 -- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d90 msgr2=0x7fecd8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.418+0000 7feceaffd700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d90 0x7fecd8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7feceaffd700 1 -- 172.21.15.53:0/819504591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80035b0 con 0x7fecd8003a00 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7feceaffd700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd8003a00 0x7fecd8003e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00c190 tx=0x7fecd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b8ec1ba649b0d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc007b70 con 0x7fecd8003a00 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc00cd30 con 0x7fecd8003a00 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7fecea7fc700 1 -- 172.21.15.53:0/819504591 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00cea0 con 0x7fecd8003a00 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7fecf19ff700 1 -- 172.21.15.53:0/819504591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd8003a00 msgr2=0x7fecd8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7fecf19ff700 1 --2- 172.21.15.53:0/819504591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd8003a00 0x7fecd8003e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00c190 tx=0x7fecd8005880 comp rx=0 tx=0).stop 2023-11-28T11:21:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.419+0000 7feceb7fe700 1 -- 172.21.15.53:0/819504591 reap_dead start 2023-11-28T11:21:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.420+0000 7fecf19ff700 1 -- 172.21.15.53:0/819504591 shutdown_connections 2023-11-28T11:21:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.420+0000 7fecf19ff700 1 -- 172.21.15.53:0/819504591 >> 172.21.15.53:0/819504591 conn(0x7fecec072600 msgr2=0x7fecec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.420+0000 7fecf19ff700 1 -- 172.21.15.53:0/819504591 shutdown_connections 2023-11-28T11:21:56.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.420+0000 7fecf19ff700 1 -- 172.21.15.53:0/819504591 wait complete. 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.421+0000 7fecf19ff700 1 Processor -- start 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.422+0000 7fecf19ff700 1 -- start start 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.422+0000 7fecf19ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd8003a00 0x7fecec1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.422+0000 7fecf19ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fecf19ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 0x7fecec15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fecf19ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec153600 con 0x7fecec150350 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fecf19ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec152020 con 0x7fecec14f980 2023-11-28T11:21:56.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fecf19ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec152300 con 0x7fecd8003a00 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49908/0 (socket says 172.21.15.53:49908) 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 -- 172.21.15.53:0/2028060843 learned_addr learned my addr 172.21.15.53:0/2028060843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7feceaffd700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd8003a00 0x7fecec1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7feceb7fe700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 0x7fecec15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 -- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd8003a00 msgr2=0x7fecec1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd8003a00 0x7fecec1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 -- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 msgr2=0x7fecec15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 0x7fecec15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 -- 172.21.15.53:0/2028060843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0d15e0 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7feceb7fe700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 0x7fecec15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.423+0000 7fece3fff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011aa0 tx=0x7fecd4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.424+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400f940 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.424+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400fac0 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.424+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400fe70 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.424+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0d1300 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.424+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec15cd80 con 0x7fecec150350 2023-11-28T11:21:56.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.425+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0745c0 con 0x7fecec150350 2023-11-28T11:21:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.426+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fecd4014440 con 0x7fecec150350 2023-11-28T11:21:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.427+0000 7fece8ff9700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 0x7feccc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.427+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fecd4086660 con 0x7fecec150350 2023-11-28T11:21:56.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.427+0000 7feceaffd700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 0x7feccc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.428+0000 7feceaffd700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 0x7feccc06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecec0ca1f0 tx=0x7fecdc00a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.431+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fecd4052e50 con 0x7fecec150350 2023-11-28T11:21:56.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.636+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecec0d08a0 con 0x7fecec150350 2023-11-28T11:21:56.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.638+0000 7fece8ff9700 1 -- 172.21.15.53:0/2028060843 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18159 (secure 0 0 0) 0x7fecd4056470 con 0x7fecec150350 2023-11-28T11:21:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:56.639 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:21:38.857031+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.7","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:56.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 msgr2=0x7feccc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 0x7feccc06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecec0ca1f0 tx=0x7fecdc00a000 comp rx=0 tx=0).stop 2023-11-28T11:21:56.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 msgr2=0x7fecec1e8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011aa0 tx=0x7fecd4011db0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 shutdown_connections 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feccc0682f0 0x7feccc06a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd8003a00 0x7fecec1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec14f980 0x7fecec15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 --2- 172.21.15.53:0/2028060843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecec150350 0x7fecec1e8100 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 >> 172.21.15.53:0/2028060843 conn(0x7fecec072600 msgr2=0x7fecec15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 shutdown_connections 2023-11-28T11:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.640+0000 7fecf19ff700 1 -- 172.21.15.53:0/2028060843 wait complete. 2023-11-28T11:21:56.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:17:40.990526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:17:41.611483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:46.350181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:47.505564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:21:56.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:56.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.863+0000 7fa516dff700 1 Processor -- start 2023-11-28T11:21:56.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.863+0000 7fa516dff700 1 -- start start 2023-11-28T11:21:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa516dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa5100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa516dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5100cd7c0 0x7fa5100cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa516dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5100ce160 con 0x7fa510161340 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa516dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5100b5800 con 0x7fa5100cd7c0 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa516dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100d1950 con 0x7fa51015c9b0 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa50ffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa510161340 0x7fa5100d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40608/0 (socket says 172.21.15.53:40608) 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.864+0000 7fa50ffff700 1 -- 172.21.15.53:0/1810576767 learned_addr learned my addr 172.21.15.53:0/1810576767 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa5100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa51539c700 1 --2- 172.21.15.53:0/1810576767 >> v2:172.21.15.157:3300/0 conn(0x7fa5100cd7c0 0x7fa5100cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799874122 0 0) 0x7fa5100ce160 con 0x7fa510161340 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5000035b0 con 0x7fa510161340 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3169233005 0 0) 0x7fa5000035b0 con 0x7fa510161340 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 msgr2=0x7fa5100b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa5100b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 >> v2:172.21.15.157:3300/0 conn(0x7fa5100cd7c0 msgr2=0x7fa5100cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 --2- 172.21.15.53:0/1810576767 >> v2:172.21.15.157:3300/0 conn(0x7fa5100cd7c0 0x7fa5100cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100d24f0 con 0x7fa510161340 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fa4fc001670 con 0x7fa510161340 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa51539c700 1 --2- 172.21.15.53:0/1810576767 >> v2:172.21.15.157:3300/0 conn(0x7fa5100cd7c0 0x7fa5100cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa5100b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.865+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 >> v1:172.21.15.53:6789/0 conn(0x7fa510161340 legacy=0x7fa5100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa500003a00 0x7fa500003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500004850 0x7fa500004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa500005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100b5800 con 0x7fa500003a00 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100d1950 con 0x7fa500004850 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50000ba10 con 0x7fa51015c9b0 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa500003a00 0x7fa500003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa51539c700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa500005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa50ffff700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500004850 0x7fa500004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 -- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 msgr2=0x7fa500005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa51015c9b0 0x7fa500005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 -- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500004850 msgr2=0x7fa500004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500004850 0x7fa500004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.866+0000 7fa514b9b700 1 -- 172.21.15.53:0/1810576767 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040094a0 con 0x7fa500003a00 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.867+0000 7fa514b9b700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa500003a00 0x7fa500003e10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d30 tx=0x7fa500005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979f09dab6065160 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.868+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009e80 con 0x7fa500003a00 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.868+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400e9a0 con 0x7fa500003a00 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.868+0000 7fa50f7fe700 1 -- 172.21.15.53:0/1810576767 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400ed10 con 0x7fa500003a00 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 -- 172.21.15.53:0/1810576767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa500003a00 msgr2=0x7fa500003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 --2- 172.21.15.53:0/1810576767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa500003a00 0x7fa500003e10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d30 tx=0x7fa500005880 comp rx=0 tx=0).stop 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa51539c700 1 -- 172.21.15.53:0/1810576767 reap_dead start 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 -- 172.21.15.53:0/1810576767 shutdown_connections 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 -- 172.21.15.53:0/1810576767 >> 172.21.15.53:0/1810576767 conn(0x7fa510072600 msgr2=0x7fa5100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 -- 172.21.15.53:0/1810576767 shutdown_connections 2023-11-28T11:21:56.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.869+0000 7fa516dff700 1 -- 172.21.15.53:0/1810576767 wait complete. 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.871+0000 7fa516dff700 1 Processor -- start 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.871+0000 7fa516dff700 1 -- start start 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500003a00 0x7fa5101e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5101e8ef0 0x7fa5101f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c8220 con 0x7fa5101f5cc0 2023-11-28T11:21:56.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101f6160 con 0x7fa500003a00 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa516dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c75e0 con 0x7fa5101e8ef0 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa51539c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa51539c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49918/0 (socket says 172.21.15.53:49918) 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa51539c700 1 -- 172.21.15.53:0/2407459400 learned_addr learned my addr 172.21.15.53:0/2407459400 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa514b9b700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500003a00 0x7fa5101e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.872+0000 7fa50ffff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5101e8ef0 0x7fa5101f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 -- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5101e8ef0 msgr2=0x7fa5101f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5101e8ef0 0x7fa5101f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 -- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500003a00 msgr2=0x7fa5101e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500003a00 0x7fa5101e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 -- 172.21.15.53:0/2407459400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100d1300 con 0x7fa5101f5cc0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa51539c700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa5100d2960 tx=0x7fa5080116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800e070 con 0x7fa5101f5cc0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa508011da0 con 0x7fa5101f5cc0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5100b5520 con 0x7fa5101f5cc0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508009ea0 con 0x7fa5101f5cc0 2023-11-28T11:21:56.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.873+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5100d15e0 con 0x7fa5101f5cc0 2023-11-28T11:21:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.874+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5100c8360 con 0x7fa5101f5cc0 2023-11-28T11:21:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.875+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa508003a20 con 0x7fa5101f5cc0 2023-11-28T11:21:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.876+0000 7fa50d7fa700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 0x7fa4f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.877+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa508086350 con 0x7fa5101f5cc0 2023-11-28T11:21:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.877+0000 7fa514b9b700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 0x7fa4f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.878+0000 7fa514b9b700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 0x7fa4f406a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5040090b0 tx=0x7fa504008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:56.880+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa5100c8360 con 0x7fa5101f5cc0 2023-11-28T11:21:57.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.082+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5100c8360 con 0x7fa5101f5cc0 2023-11-28T11:21:57.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.083+0000 7fa50d7fa700 1 -- 172.21.15.53:0/2407459400 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5100c8360 con 0x7fa5101f5cc0 2023-11-28T11:21:57.084 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-28T11:21:57.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 msgr2=0x7fa4f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 0x7fa4f406a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5040090b0 tx=0x7fa504008030 comp rx=0 tx=0).stop 2023-11-28T11:21:57.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 msgr2=0x7fa5101e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa5100d2960 tx=0x7fa5080116e0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 shutdown_connections 2023-11-28T11:21:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fa4f40682f0 0x7fa4f406a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5101f5cc0 0x7fa5101e6620 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa500003a00 0x7fa5101e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 --2- 172.21.15.53:0/2407459400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5101e8ef0 0x7fa5101f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 >> 172.21.15.53:0/2407459400 conn(0x7fa510072600 msgr2=0x7fa5100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.085+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 shutdown_connections 2023-11-28T11:21:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.086+0000 7fa516dff700 1 -- 172.21.15.53:0/2407459400 wait complete. 2023-11-28T11:21:57.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:57.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.309+0000 7fba52dff700 1 Processor -- start 2023-11-28T11:21:57.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.309+0000 7fba52dff700 1 -- start start 2023-11-28T11:21:57.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba52dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 0x7fba4c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba52dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba52dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4c0d1950 con 0x7fba4c161340 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba52dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4c0b5800 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba52dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4c0ce140 con 0x7fba4c15c9b0 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49934/0 (socket says 172.21.15.53:49934) 2023-11-28T11:21:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 -- 172.21.15.53:0/3959368538 learned_addr learned my addr 172.21.15.53:0/3959368538 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba50b9b700 1 --2- 172.21.15.53:0/3959368538 >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 0x7fba4c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 -- 172.21.15.53:0/3959368538 >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 msgr2=0x7fba4c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 --2- 172.21.15.53:0/3959368538 >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 0x7fba4c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 -- 172.21.15.53:0/3959368538 >> v1:172.21.15.157:6789/0 conn(0x7fba4c161340 legacy=0x7fba4c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba5139c700 1 -- 172.21.15.53:0/3959368538 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0d24f0 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.311+0000 7fba50b9b700 1 --2- 172.21.15.53:0/3959368538 >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 0x7fba4c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba5139c700 1 --2- 172.21.15.53:0/3959368538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fba4c067460 tx=0x7fba4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e314785f1d9a5f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba4b7fe700 1 -- 172.21.15.53:0/3959368538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400e070 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba4b7fe700 1 -- 172.21.15.53:0/3959368538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4400b9c0 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba4b7fe700 1 -- 172.21.15.53:0/3959368538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400bd70 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba52dff700 1 -- 172.21.15.53:0/3959368538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 msgr2=0x7fba4c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.312+0000 7fba52dff700 1 --2- 172.21.15.53:0/3959368538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fba4c067460 tx=0x7fba4400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 -- 172.21.15.53:0/3959368538 shutdown_connections 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 --2- 172.21.15.53:0/3959368538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cdc00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 --2- 172.21.15.53:0/3959368538 >> v2:172.21.15.157:3300/0 conn(0x7fba4c15c9b0 0x7fba4c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 -- 172.21.15.53:0/3959368538 >> 172.21.15.53:0/3959368538 conn(0x7fba4c072600 msgr2=0x7fba4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 -- 172.21.15.53:0/3959368538 shutdown_connections 2023-11-28T11:21:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.313+0000 7fba52dff700 1 -- 172.21.15.53:0/3959368538 wait complete. 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.314+0000 7fba52dff700 1 Processor -- start 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.315+0000 7fba52dff700 1 -- start start 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c15c9b0 0x7fba4c1e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c161340 0x7fba4c1f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c0cc7b0 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c0cb920 con 0x7fba4c161340 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba52dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c1e7ba0 con 0x7fba4c15c9b0 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49946/0 (socket says 172.21.15.53:49946) 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 -- 172.21.15.53:0/1323390634 learned_addr learned my addr 172.21.15.53:0/1323390634 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba4bfff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c15c9b0 0x7fba4c1e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba5139c700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c161340 0x7fba4c1f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 -- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c15c9b0 msgr2=0x7fba4c1e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c15c9b0 0x7fba4c1e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 -- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c161340 msgr2=0x7fba4c1f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c161340 0x7fba4c1f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.316+0000 7fba50b9b700 1 -- 172.21.15.53:0/1323390634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009b30 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba50b9b700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e760 tx=0x7fba4000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40013610 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40013770 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4c0d1300 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40013b20 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.317+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4c0b5520 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.318+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4c0d15e0 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.319+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba40023070 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.320+0000 7fba497fa700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 0x7fba3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.320+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fba40085910 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.321+0000 7fba4bfff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 0x7fba3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.321+0000 7fba4bfff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 0x7fba3006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba4c1f59a0 tx=0x7fba38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.324+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba400520e0 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.524+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba4c0cca20 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.524+0000 7fba497fa700 1 -- 172.21.15.53:0/1323390634 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v63) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba40055700 con 0x7fba4c0cd7c0 2023-11-28T11:21:57.526 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 8 2023-11-28T11:21:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 msgr2=0x7fba3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 0x7fba3006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba4c1f59a0 tx=0x7fba38009040 comp rx=0 tx=0).stop 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 msgr2=0x7fba4c0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e760 tx=0x7fba4000eb20 comp rx=0 tx=0).stop 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 shutdown_connections 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fba300682f0 0x7fba3006a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c161340 0x7fba4c1f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c0cd7c0 0x7fba4c0cb3e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 --2- 172.21.15.53:0/1323390634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c15c9b0 0x7fba4c1e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 >> 172.21.15.53:0/1323390634 conn(0x7fba4c072600 msgr2=0x7fba4c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 shutdown_connections 2023-11-28T11:21:57.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.527+0000 7fba52dff700 1 -- 172.21.15.53:0/1323390634 wait complete. 2023-11-28T11:21:57.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:57.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.751+0000 7f8443fff700 1 Processor -- start 2023-11-28T11:21:57.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.751+0000 7f8443fff700 1 -- start start 2023-11-28T11:21:57.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f8443fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f843c15c9b0 0x7f843c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f8443fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f8443fff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0d1950 con 0x7f843c0d2320 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f8443fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0b5800 con 0x7f843c15c9b0 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f8443fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b5520 con 0x7f843c161340 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f844259c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f843c0d2320 0x7f843c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40624/0 (socket says 172.21.15.53:40624) 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.752+0000 7f844259c700 1 -- 172.21.15.53:0/1119960991 learned_addr learned my addr 172.21.15.53:0/1119960991 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 --2- 172.21.15.53:0/1119960991 >> v2:172.21.15.157:3300/0 conn(0x7f843c15c9b0 0x7f843c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f844159a700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8440d99700 1 -- 172.21.15.53:0/1119960991 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689512462 0 0) 0x7f843c0d1950 con 0x7f843c0d2320 2023-11-28T11:21:57.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8440d99700 1 -- 172.21.15.53:0/1119960991 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84280035b0 con 0x7f843c0d2320 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 -- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 msgr2=0x7f843c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 -- 172.21.15.53:0/1119960991 >> v1:172.21.15.53:6789/0 conn(0x7f843c0d2320 legacy=0x7f843c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 -- 172.21.15.53:0/1119960991 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0d1300 con 0x7f843c15c9b0 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f844159a700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.753+0000 7f8441d9b700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b4bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8438009840 tx=0x7f8438009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a2fd3880b37cc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8440d99700 1 -- 172.21.15.53:0/1119960991 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011830 con 0x7f843c15c9b0 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8440d99700 1 -- 172.21.15.53:0/1119960991 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8438011990 con 0x7f843c15c9b0 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8440d99700 1 -- 172.21.15.53:0/1119960991 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011d00 con 0x7f843c15c9b0 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 -- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 msgr2=0x7f843c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b4bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8438009840 tx=0x7f8438009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 -- 172.21.15.53:0/1119960991 shutdown_connections 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b4bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 --2- 172.21.15.53:0/1119960991 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 -- 172.21.15.53:0/1119960991 >> 172.21.15.53:0/1119960991 conn(0x7f843c072600 msgr2=0x7f843c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 -- 172.21.15.53:0/1119960991 shutdown_connections 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.754+0000 7f8443fff700 1 -- 172.21.15.53:0/1119960991 wait complete. 2023-11-28T11:21:57.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.756+0000 7f8443fff700 1 Processor -- start 2023-11-28T11:21:57.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.756+0000 7f8443fff700 1 -- start start 2023-11-28T11:21:57.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c151570 con 0x7f843c0d2320 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c146e00 con 0x7f843c15c9b0 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8443fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c157670 con 0x7f843c161340 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8441d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.757+0000 7f8441d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49962/0 (socket says 172.21.15.53:49962) 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 -- 172.21.15.53:0/2121417260 learned_addr learned my addr 172.21.15.53:0/2121417260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 -- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 msgr2=0x7f843c0b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f844159a700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 -- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 msgr2=0x7f843c0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 -- 172.21.15.53:0/2121417260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0094a0 con 0x7f843c0d2320 2023-11-28T11:21:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f844159a700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8441d9b700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b90 tx=0x7f8438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84380036a0 con 0x7f843c0d2320 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843801e070 con 0x7f843c0d2320 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438003a60 con 0x7f843c0d2320 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.758+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84380094a0 con 0x7f843c0d2320 2023-11-28T11:21:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.759+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c0b5130 con 0x7f843c0d2320 2023-11-28T11:21:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.760+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0d15e0 con 0x7f843c0d2320 2023-11-28T11:21:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.760+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f843800f050 con 0x7f843c0d2320 2023-11-28T11:21:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.761+0000 7f84327fc700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 0x7f842006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.761+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8438086d90 con 0x7f843c0d2320 2023-11-28T11:21:57.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.762+0000 7f844159a700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 0x7f842006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.763+0000 7f844159a700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 0x7f842006a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f843c1f6560 tx=0x7f842c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.765+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8438053470 con 0x7f843c0d2320 2023-11-28T11:21:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.969+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f843c0b52c0 con 0x7f843c0d2320 2023-11-28T11:21:57.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.970+0000 7f84327fc700 1 -- 172.21.15.53:0/2121417260 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84380150a0 con 0x7f843c0d2320 2023-11-28T11:21:57.971 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-28T11:21:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 msgr2=0x7f842006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 0x7f842006a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f843c1f6560 tx=0x7f842c008040 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 msgr2=0x7f843c1468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b90 tx=0x7f8438004750 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 shutdown_connections 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f84200682f0 0x7f842006a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c161340 0x7f843c0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c0d2320 0x7f843c1468c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 --2- 172.21.15.53:0/2121417260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c15c9b0 0x7f843c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 >> 172.21.15.53:0/2121417260 conn(0x7f843c072600 msgr2=0x7f843c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 shutdown_connections 2023-11-28T11:21:57.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:57.972+0000 7f8443fff700 1 -- 172.21.15.53:0/2121417260 wait complete. 2023-11-28T11:21:57.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:58.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.196+0000 7f561ddff700 1 Processor -- start 2023-11-28T11:21:58.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.196+0000 7f561ddff700 1 -- start start 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f561ddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f561ddff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56180d2320 0x7f56180cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f561ddff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56180d1950 con 0x7f561815c9b0 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f561ddff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56180b5800 con 0x7f56180d2320 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f561ddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56180b5300 con 0x7f5618161340 2023-11-28T11:21:58.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56180d2320 0x7f56180cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56180d2320 0x7f56180cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49968/0 (socket says 172.21.15.53:49968) 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 -- 172.21.15.53:0/2168878050 learned_addr learned my addr 172.21.15.53:0/2168878050 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5616ffd700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 -- 172.21.15.53:0/2168878050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 msgr2=0x7f56180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 -- 172.21.15.53:0/2168878050 >> v1:172.21.15.157:6789/0 conn(0x7f561815c9b0 legacy=0x7f56180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.198+0000 7f5617fff700 1 -- 172.21.15.53:0/2168878050 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56180b5520 con 0x7f56180d2320 2023-11-28T11:21:58.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f5617fff700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56180d2320 0x7f56180cd7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5610009e80 tx=0x7f561000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff87378e0eabb9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f56167fc700 1 -- 172.21.15.53:0/2168878050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000e070 con 0x7f56180d2320 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f56167fc700 1 -- 172.21.15.53:0/2168878050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561000b980 con 0x7f56180d2320 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f56167fc700 1 -- 172.21.15.53:0/2168878050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000bd30 con 0x7f56180d2320 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f561ddff700 1 -- 172.21.15.53:0/2168878050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56180d2320 msgr2=0x7f56180cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.199+0000 7f561ddff700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56180d2320 0x7f56180cd7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5610009e80 tx=0x7f561000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 -- 172.21.15.53:0/2168878050 shutdown_connections 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56180d2320 0x7f56180cd7c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 --2- 172.21.15.53:0/2168878050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 -- 172.21.15.53:0/2168878050 >> 172.21.15.53:0/2168878050 conn(0x7f5618072600 msgr2=0x7f56180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 -- 172.21.15.53:0/2168878050 shutdown_connections 2023-11-28T11:21:58.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.200+0000 7f561ddff700 1 -- 172.21.15.53:0/2168878050 wait complete. 2023-11-28T11:21:58.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.201+0000 7f561ddff700 1 Processor -- start 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 -- start start 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56180d2320 0x7f5618150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56181e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181e7e80 con 0x7f561815c9b0 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180c85b0 con 0x7f56180d2320 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.202+0000 7f561ddff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56180c8890 con 0x7f5618161340 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49984/0 (socket says 172.21.15.53:49984) 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 -- 172.21.15.53:0/3136028018 learned_addr learned my addr 172.21.15.53:0/3136028018 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5617fff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56181e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f56177fe700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56180d2320 0x7f5618150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 -- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 msgr2=0x7f56181e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56181e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 -- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56180d2320 msgr2=0x7f5618150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56180d2320 0x7f5618150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 -- 172.21.15.53:0/3136028018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610009b30 con 0x7f561815c9b0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.203+0000 7f5616ffd700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f560c00e810 tx=0x7f560c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.204+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c00c070 con 0x7f561815c9b0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.204+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c013610 con 0x7f561815c9b0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.204+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56180d15e0 con 0x7f561815c9b0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.204+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c0139c0 con 0x7f561815c9b0 2023-11-28T11:21:58.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.204+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56180d1300 con 0x7f561815c9b0 2023-11-28T11:21:58.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.205+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561815cd80 con 0x7f561815c9b0 2023-11-28T11:21:58.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.206+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f560c013b20 con 0x7f561815c9b0 2023-11-28T11:21:58.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.207+0000 7f55fffff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 0x7f560406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.207+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f560c085a00 con 0x7f561815c9b0 2023-11-28T11:21:58.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.207+0000 7f56177fe700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 0x7f560406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.208+0000 7f56177fe700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 0x7f560406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5608007fd0 tx=0x7f5608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.211+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f560c052170 con 0x7f561815c9b0 2023-11-28T11:21:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.415+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56180745c0 con 0x7f561815c9b0 2023-11-28T11:21:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.415+0000 7f55fffff700 1 -- 172.21.15.53:0/3136028018 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f560c0170a0 con 0x7f561815c9b0 2023-11-28T11:21:58.416 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 msgr2=0x7f560406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 0x7f560406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5608007fd0 tx=0x7f5608009040 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 msgr2=0x7f561814fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f560c00e810 tx=0x7f560c00eb20 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 shutdown_connections 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f56040682f0 0x7f560406a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5618161340 0x7f56181e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56180d2320 0x7f5618150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 --2- 172.21.15.53:0/3136028018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f561815c9b0 0x7f561814fed0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.417+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 >> 172.21.15.53:0/3136028018 conn(0x7f5618072600 msgr2=0x7f56180b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.418+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 shutdown_connections 2023-11-28T11:21:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.418+0000 7f561ddff700 1 -- 172.21.15.53:0/3136028018 wait complete. 2023-11-28T11:21:58.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.638+0000 7fb6edbff700 1 Processor -- start 2023-11-28T11:21:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.638+0000 7fb6edbff700 1 -- start start 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.639+0000 7fb6edbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.639+0000 7fb6edbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e80d1950 con 0x7fb6e8161340 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.639+0000 7fb6edbff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e80d24f0 con 0x7fb6e80cd7c0 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.639+0000 7fb6edbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80b5800 con 0x7fb6e815c9b0 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e6ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb6e8161340 0x7fb6e80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40634/0 (socket says 172.21.15.53:40634) 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e6ffd700 1 -- 172.21.15.53:0/2775628962 learned_addr learned my addr 172.21.15.53:0/2775628962 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e77fe700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90908677 0 0) 0x7fb6e80d1950 con 0x7fb6e8161340 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d00035b0 con 0x7fb6e8161340 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1859304195 0 0) 0x7fb6d00035b0 con 0x7fb6e8161340 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 msgr2=0x7fb6e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6e80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 >> v1:172.21.15.157:6789/0 conn(0x7fb6e80cd7c0 legacy=0x7fb6e80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80b5520 con 0x7fb6e8161340 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.640+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fb6dc001670 con 0x7fb6e8161340 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 >> v1:172.21.15.53:6789/0 conn(0x7fb6e8161340 legacy=0x7fb6e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d0004d90 0x7fb6d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80b5800 con 0x7fb6d0004d90 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d000baf0 con 0x7fb6d0003f70 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d000bcb0 con 0x7fb6e815c9b0 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d0004d90 0x7fb6d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e6ffd700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e77fe700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 -- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 msgr2=0x7fb6d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e815c9b0 0x7fb6d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 -- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0003f70 msgr2=0x7fb6d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.641+0000 7fb6e7fff700 1 -- 172.21.15.53:0/2775628962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00035b0 con 0x7fb6d0004d90 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6e7fff700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d0004d90 0x7fb6d0005200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c260 tx=0x7fb6d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ffc425494f5083 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0001e70 con 0x7fb6d0004d90 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e000ce20 con 0x7fb6d0004d90 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6e67fc700 1 -- 172.21.15.53:0/2775628962 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0012660 con 0x7fb6d0004d90 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6edbff700 1 -- 172.21.15.53:0/2775628962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d0004d90 msgr2=0x7fb6d0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6edbff700 1 --2- 172.21.15.53:0/2775628962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d0004d90 0x7fb6d0005200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c260 tx=0x7fb6d0005880 comp rx=0 tx=0).stop 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6e7fff700 1 -- 172.21.15.53:0/2775628962 reap_dead start 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6edbff700 1 -- 172.21.15.53:0/2775628962 shutdown_connections 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.642+0000 7fb6edbff700 1 -- 172.21.15.53:0/2775628962 >> 172.21.15.53:0/2775628962 conn(0x7fb6e8072600 msgr2=0x7fb6e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.643+0000 7fb6edbff700 1 -- 172.21.15.53:0/2775628962 shutdown_connections 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.643+0000 7fb6edbff700 1 -- 172.21.15.53:0/2775628962 wait complete. 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.644+0000 7fb6edbff700 1 Processor -- start 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.644+0000 7fb6edbff700 1 -- start start 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0004d90 0x7fb6e80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e81fa1f0 0x7fb6e81fa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81fabb0 con 0x7fb6e81faeb0 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8153d30 con 0x7fb6d0004d90 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.645+0000 7fb6edbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80b4030 con 0x7fb6e81fa1f0 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50002/0 (socket says 172.21.15.53:50002) 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 -- 172.21.15.53:0/3371167970 learned_addr learned my addr 172.21.15.53:0/3371167970 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e77fe700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0004d90 0x7fb6e80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e7fff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e81fa1f0 0x7fb6e81fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 -- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e81fa1f0 msgr2=0x7fb6e81fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e81fa1f0 0x7fb6e81fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 -- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0004d90 msgr2=0x7fb6e80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0004d90 0x7fb6e80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 -- 172.21.15.53:0/3371167970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80d15e0 con 0x7fb6e81faeb0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.646+0000 7fb6e6ffd700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007290 tx=0x7fb6dc0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.647+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc001650 con 0x7fb6e81faeb0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.647+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00f660 con 0x7fb6e81faeb0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.647+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00f9f0 con 0x7fb6e81faeb0 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.647+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80d1300 con 0x7fb6e81faeb0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.647+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e815cd80 con 0x7fb6e81faeb0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.648+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e80745c0 con 0x7fb6e81faeb0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.649+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb6dc00fb50 con 0x7fb6e81faeb0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.650+0000 7fb6cffff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 0x7fb6d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.650+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb6dc085d90 con 0x7fb6e81faeb0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.650+0000 7fb6e77fe700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 0x7fb6d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.651+0000 7fb6e77fe700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 0x7fb6d406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009c60 tx=0x7fb6d8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.654+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb6dc052470 con 0x7fb6e81faeb0 2023-11-28T11:21:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.903+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6e80b4170 con 0x7fb6e81faeb0 2023-11-28T11:21:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.903+0000 7fb6cffff700 1 -- 172.21.15.53:0/3371167970 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fb6dc055a90 con 0x7fb6e81faeb0 2023-11-28T11:21:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:58.905 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:21:04.863310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:21:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 msgr2=0x7fb6d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 0x7fb6d406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009c60 tx=0x7fb6d8009210 comp rx=0 tx=0).stop 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 msgr2=0x7fb6e81537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007290 tx=0x7fb6dc0075a0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 shutdown_connections 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb6d40682f0 0x7fb6d406a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6e81fa1f0 0x7fb6e81fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d0004d90 0x7fb6e80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 --2- 172.21.15.53:0/3371167970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e81faeb0 0x7fb6e81537f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 >> 172.21.15.53:0/3371167970 conn(0x7fb6e8072600 msgr2=0x7fb6e815eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 shutdown_connections 2023-11-28T11:21:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:58.906+0000 7fb6edbff700 1 -- 172.21.15.53:0/3371167970 wait complete. 2023-11-28T11:21:58.909 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 33 2023-11-28T11:21:58.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:59.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.125+0000 7f3bbc9ff700 1 Processor -- start 2023-11-28T11:21:59.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.125+0000 7f3bbc9ff700 1 -- start start 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bbc9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3bb8161340 0x7f3bb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bbc9ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 0x7f3bb80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bbc9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb80d1950 con 0x7f3bb815c9b0 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bbc9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb80b5800 con 0x7f3bb8161340 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bbc9ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb80b5300 con 0x7f3bb80d2320 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3bb8161340 0x7f3bb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3bb8161340 0x7f3bb80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50014/0 (socket says 172.21.15.53:50014) 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 -- 172.21.15.53:0/3729976744 learned_addr learned my addr 172.21.15.53:0/3729976744 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:59.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bb6d9d700 1 --2- 172.21.15.53:0/3729976744 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 0x7f3bb80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 -- 172.21.15.53:0/3729976744 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 msgr2=0x7f3bb80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 --2- 172.21.15.53:0/3729976744 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 0x7f3bb80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 -- 172.21.15.53:0/3729976744 >> v1:172.21.15.157:6789/0 conn(0x7f3bb815c9b0 legacy=0x7f3bb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3baffff700 1 -- 172.21.15.53:0/3729976744 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80b5520 con 0x7f3bb8161340 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.127+0000 7f3bb6d9d700 1 --2- 172.21.15.53:0/3729976744 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 0x7f3bb80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3baffff700 1 --2- 172.21.15.53:0/3729976744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8161340 0x7f3bb80d1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806c170 tx=0x7f3ba0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a077851b8be5f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bb5d9b700 1 -- 172.21.15.53:0/3729976744 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000b860 con 0x7f3bb8161340 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bb5d9b700 1 -- 172.21.15.53:0/3729976744 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba000b9c0 con 0x7f3bb8161340 2023-11-28T11:21:59.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bb5d9b700 1 -- 172.21.15.53:0/3729976744 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000bd50 con 0x7f3bb8161340 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/3729976744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8161340 msgr2=0x7f3bb80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/3729976744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8161340 0x7f3bb80d1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806c170 tx=0x7f3ba0009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/3729976744 shutdown_connections 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/3729976744 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80d2320 0x7f3bb80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/3729976744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8161340 0x7f3bb80d1de0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.128+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/3729976744 >> 172.21.15.53:0/3729976744 conn(0x7f3bb8072600 msgr2=0x7f3bb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.129+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/3729976744 shutdown_connections 2023-11-28T11:21:59.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.129+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/3729976744 wait complete. 2023-11-28T11:21:59.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.130+0000 7f3bbc9ff700 1 Processor -- start 2023-11-28T11:21:59.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.130+0000 7f3bbc9ff700 1 -- start start 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb815c9b0 0x7f3bb80cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 0x7f3bb815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80cce90 con 0x7f3bb80d2320 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80cd170 con 0x7f3bb815c9b0 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.131+0000 7f3bbc9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80cd450 con 0x7f3bb8161340 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50030/0 (socket says 172.21.15.53:50030) 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 -- 172.21.15.53:0/4269717479 learned_addr learned my addr 172.21.15.53:0/4269717479 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3baffff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb815c9b0 0x7f3bb80cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb6d9d700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 0x7f3bb815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 -- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 msgr2=0x7f3bb815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 0x7f3bb815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 -- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb815c9b0 msgr2=0x7f3bb80cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb815c9b0 0x7f3bb80cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 -- 172.21.15.53:0/4269717479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0009550 con 0x7f3bb80d2320 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb6d9d700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 0x7f3bb815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.132+0000 7f3bb659c700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c450 tx=0x7f3ba800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.133+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80052d0 con 0x7f3bb80d2320 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.133+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba8005430 con 0x7f3bb80d2320 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.133+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb80d15e0 con 0x7f3bb80d2320 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.133+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80038c0 con 0x7f3bb80d2320 2023-11-28T11:21:59.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.133+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb80d1300 con 0x7f3bb80d2320 2023-11-28T11:21:59.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.134+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb815cd80 con 0x7f3bb80d2320 2023-11-28T11:21:59.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.135+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ba8019030 con 0x7f3bb80d2320 2023-11-28T11:21:59.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.136+0000 7f3baeffd700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 0x7f3b9806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.136+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3ba8084f20 con 0x7f3bb80d2320 2023-11-28T11:21:59.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.136+0000 7f3baffff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 0x7f3b9806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.137+0000 7f3baffff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 0x7f3b9806a7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000c00 tx=0x7f3ba0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.140+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ba8051790 con 0x7f3bb80d2320 2023-11-28T11:21:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.345+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bb80745c0 con 0x7f3bb80d2320 2023-11-28T11:21:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.346+0000 7f3baeffd700 1 -- 172.21.15.53:0/4269717479 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18159 (secure 0 0 0) 0x7f3ba8017090 con 0x7f3bb80d2320 2023-11-28T11:21:59.348 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:59.348 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:21:38.857031+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.7","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:59.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 msgr2=0x7f3b9806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 0x7f3b9806a7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000c00 tx=0x7f3ba0008030 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 msgr2=0x7f3bb80cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c450 tx=0x7f3ba800c760 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 shutdown_connections 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b98068340 0x7f3b9806a7f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8161340 0x7f3bb815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb80d2320 0x7f3bb80cac70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 --2- 172.21.15.53:0/4269717479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb815c9b0 0x7f3bb80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.348+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 >> 172.21.15.53:0/4269717479 conn(0x7f3bb8072600 msgr2=0x7f3bb806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.349+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 shutdown_connections 2023-11-28T11:21:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.349+0000 7f3bbc9ff700 1 -- 172.21.15.53:0/4269717479 wait complete. 2023-11-28T11:21:59.368 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:21:59.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.569+0000 7f122fbff700 1 Processor -- start 2023-11-28T11:21:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.569+0000 7f122fbff700 1 -- start start 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 0x7f12280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1228161340 0x7f12280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12280d2320 0x7f12280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12280b5800 con 0x7f1228161340 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12280d1950 con 0x7f122815c9b0 2023-11-28T11:21:59.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122fbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12280b5520 con 0x7f12280d2320 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122d19a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1228161340 0x7f12280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122d19a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1228161340 0x7f12280d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50038/0 (socket says 172.21.15.53:50038) 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122d19a700 1 -- 172.21.15.53:0/3388645313 learned_addr learned my addr 172.21.15.53:0/3388645313 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122e19c700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12280d2320 0x7f12280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.570+0000 7f122d99b700 1 --2- 172.21.15.53:0/3388645313 >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 0x7f12280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 -- 172.21.15.53:0/3388645313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12280d2320 msgr2=0x7f12280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12280d2320 0x7f12280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 -- 172.21.15.53:0/3388645313 >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 msgr2=0x7f12280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 --2- 172.21.15.53:0/3388645313 >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 0x7f12280b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 -- 172.21.15.53:0/3388645313 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280d1300 con 0x7f1228161340 2023-11-28T11:21:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d99b700 1 --2- 172.21.15.53:0/3388645313 >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 0x7f12280b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122d19a700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1228161340 0x7f12280d1de0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1228067460 tx=0x7f1218009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909d12c42e2a4e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122c999700 1 -- 172.21.15.53:0/3388645313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011830 con 0x7f1228161340 2023-11-28T11:21:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122c999700 1 -- 172.21.15.53:0/3388645313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218011990 con 0x7f1228161340 2023-11-28T11:21:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.571+0000 7f122c999700 1 -- 172.21.15.53:0/3388645313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011d20 con 0x7f1228161340 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 -- 172.21.15.53:0/3388645313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1228161340 msgr2=0x7f12280d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1228161340 0x7f12280d1de0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1228067460 tx=0x7f1218009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 -- 172.21.15.53:0/3388645313 shutdown_connections 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12280d2320 0x7f12280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 --2- 172.21.15.53:0/3388645313 >> v2:172.21.15.157:3300/0 conn(0x7f122815c9b0 0x7f12280b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 --2- 172.21.15.53:0/3388645313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1228161340 0x7f12280d1de0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 -- 172.21.15.53:0/3388645313 >> 172.21.15.53:0/3388645313 conn(0x7f1228072600 msgr2=0x7f12280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 -- 172.21.15.53:0/3388645313 shutdown_connections 2023-11-28T11:21:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.572+0000 7f122fbff700 1 -- 172.21.15.53:0/3388645313 wait complete. 2023-11-28T11:21:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.574+0000 7f122fbff700 1 Processor -- start 2023-11-28T11:21:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.575+0000 7f122fbff700 1 -- start start 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 0x7f1228151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 0x7f12280c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280c9090 con 0x7f12280d2320 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280c81a0 con 0x7f1228161340 2023-11-28T11:21:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122fbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12280c8480 con 0x7f122815c9b0 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122d99b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122d99b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50040/0 (socket says 172.21.15.53:50040) 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122d99b700 1 -- 172.21.15.53:0/2196539861 learned_addr learned my addr 172.21.15.53:0/2196539861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122e19c700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 0x7f12280c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.576+0000 7f122d19a700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 0x7f1228151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 -- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 msgr2=0x7f1228151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 0x7f1228151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 -- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 msgr2=0x7f12280c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 0x7f12280c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 -- 172.21.15.53:0/2196539861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218009550 con 0x7f12280d2320 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122e19c700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 0x7f12280c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d19a700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 0x7f1228151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122d99b700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f122400e760 tx=0x7f122400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224013610 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224013770 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12280b5130 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224013b20 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.577+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12280d15e0 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.578+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12280b52c0 con 0x7f12280d2320 2023-11-28T11:21:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.579+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1224023070 con 0x7f12280d2320 2023-11-28T11:21:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.580+0000 7f121e7fc700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 0x7f120c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.580+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1224085910 con 0x7f12280d2320 2023-11-28T11:21:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.581+0000 7f122d19a700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 0x7f120c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.581+0000 7f122d19a700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 0x7f120c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1228152030 tx=0x7f1218008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.584+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12240520e0 con 0x7f12280d2320 2023-11-28T11:21:59.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.830+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12280c9370 con 0x7f12280d2320 2023-11-28T11:21:59.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.831+0000 7f121e7fc700 1 -- 172.21.15.53:0/2196539861 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f1224055700 con 0x7f12280d2320 2023-11-28T11:21:59.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:21:59.832 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:21:04.863310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 msgr2=0x7f120c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 0x7f120c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1228152030 tx=0x7f1218008030 comp rx=0 tx=0).stop 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 msgr2=0x7f12280c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f122400e760 tx=0x7f122400eb20 comp rx=0 tx=0).stop 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 shutdown_connections 2023-11-28T11:21:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f120c0682f0 0x7f120c06a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228161340 0x7f12280c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12280d2320 0x7f12280c9520 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 --2- 172.21.15.53:0/2196539861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f122815c9b0 0x7f1228151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 >> 172.21.15.53:0/2196539861 conn(0x7f1228072600 msgr2=0x7f12280b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 shutdown_connections 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:21:59.833+0000 7f122fbff700 1 -- 172.21.15.53:0/2196539861 wait complete. 2023-11-28T11:21:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 33 2023-11-28T11:21:59.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:00.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.055+0000 7f2d7ebff700 1 Processor -- start 2023-11-28T11:22:00.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.056+0000 7f2d7ebff700 1 -- start start 2023-11-28T11:22:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7ebff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d78161340 0x7f2d780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7ebff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d780d2320 0x7f2d780cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7ebff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780d1950 con 0x7f2d7815c9b0 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7ebff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780b5800 con 0x7f2d780d2320 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7ebff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780b5300 con 0x7f2d78161340 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7d19c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d780d2320 0x7f2d780cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7c99b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2d7815c9b0 0x7f2d780b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40646/0 (socket says 172.21.15.53:40646) 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d7c99b700 1 -- 172.21.15.53:0/1466026436 learned_addr learned my addr 172.21.15.53:0/1466026436 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.057+0000 7f2d77fff700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d78161340 0x7f2d780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d7d19c700 1 -- 172.21.15.53:0/1466026436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d78161340 msgr2=0x7f2d780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d7d19c700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d78161340 0x7f2d780d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d7d19c700 1 -- 172.21.15.53:0/1466026436 >> v1:172.21.15.53:6789/0 conn(0x7f2d7815c9b0 legacy=0x7f2d780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d7d19c700 1 -- 172.21.15.53:0/1466026436 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780b5520 con 0x7f2d780d2320 2023-11-28T11:22:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d7d19c700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d780d2320 0x7f2d780cd7c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009dc0 tx=0x7f2d7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee7b50f20a3c3ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d777fe700 1 -- 172.21.15.53:0/1466026436 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e070 con 0x7f2d780d2320 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d777fe700 1 -- 172.21.15.53:0/1466026436 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000bae0 con 0x7f2d780d2320 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.058+0000 7f2d777fe700 1 -- 172.21.15.53:0/1466026436 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000be90 con 0x7f2d780d2320 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 -- 172.21.15.53:0/1466026436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d780d2320 msgr2=0x7f2d780cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d780d2320 0x7f2d780cd7c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009dc0 tx=0x7f2d7000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 -- 172.21.15.53:0/1466026436 shutdown_connections 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d780d2320 0x7f2d780cd7c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/1466026436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d78161340 0x7f2d780d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 -- 172.21.15.53:0/1466026436 >> 172.21.15.53:0/1466026436 conn(0x7f2d78072600 msgr2=0x7f2d780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 -- 172.21.15.53:0/1466026436 shutdown_connections 2023-11-28T11:22:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.059+0000 7f2d7ebff700 1 -- 172.21.15.53:0/1466026436 wait complete. 2023-11-28T11:22:00.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.061+0000 7f2d7ebff700 1 Processor -- start 2023-11-28T11:22:00.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.061+0000 7f2d7ebff700 1 -- start start 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 0x7f2d78151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d78161340 0x7f2d780c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781531e0 con 0x7f2d7815c9b0 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78152560 con 0x7f2d78161340 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7ebff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78152840 con 0x7f2d780d2320 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50058/0 (socket says 172.21.15.53:50058) 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 -- 172.21.15.53:0/2070524358 learned_addr learned my addr 172.21.15.53:0/2070524358 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7c99b700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 0x7f2d78151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7d19c700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d78161340 0x7f2d780c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 -- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 msgr2=0x7f2d78151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 0x7f2d78151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 -- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d78161340 msgr2=0x7f2d780c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d78161340 0x7f2d780c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d77fff700 1 -- 172.21.15.53:0/2070524358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009a70 con 0x7f2d7815c9b0 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.062+0000 7f2d7c99b700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 0x7f2d78151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d77fff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9b0 tx=0x7f2d6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64009710 con 0x7f2d7815c9b0 2023-11-28T11:22:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400c850 con 0x7f2d7815c9b0 2023-11-28T11:22:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d780d15e0 con 0x7f2d7815c9b0 2023-11-28T11:22:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cbc0 con 0x7f2d7815c9b0 2023-11-28T11:22:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.063+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780d1300 con 0x7f2d7815c9b0 2023-11-28T11:22:00.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.064+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7815cd80 con 0x7f2d7815c9b0 2023-11-28T11:22:00.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.065+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d64019030 con 0x7f2d7815c9b0 2023-11-28T11:22:00.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.066+0000 7f2d757fa700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 0x7f2d5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.066+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2d64085e90 con 0x7f2d7815c9b0 2023-11-28T11:22:00.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.067+0000 7f2d7c99b700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 0x7f2d5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.067+0000 7f2d7c99b700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 0x7f2d5c06a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c007fd0 tx=0x7f2d6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.070+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d640524f0 con 0x7f2d7815c9b0 2023-11-28T11:22:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.315+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d78153320 con 0x7f2d7815c9b0 2023-11-28T11:22:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.316+0000 7f2d757fa700 1 -- 172.21.15.53:0/2070524358 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f2d64017090 con 0x7f2d7815c9b0 2023-11-28T11:22:00.317 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:00.317 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:48.508614+0000","modified":"2023-11-28T11:21:04.863310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55331":{"gid":55331,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6834/1970232100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1970232100},{"type":"v1","addr":"172.21.15.53:6834","nonce":1970232100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:22:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 msgr2=0x7f2d5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 0x7f2d5c06a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c007fd0 tx=0x7f2d6c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 msgr2=0x7f2d780c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9b0 tx=0x7f2d6400ed70 comp rx=0 tx=0).stop 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 shutdown_connections 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2d5c0682f0 0x7f2d5c06a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d780d2320 0x7f2d78151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d78161340 0x7f2d780c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 --2- 172.21.15.53:0/2070524358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7815c9b0 0x7f2d780c8e30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 >> 172.21.15.53:0/2070524358 conn(0x7f2d78072600 msgr2=0x7f2d7815fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 shutdown_connections 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.318+0000 7f2d7ebff700 1 -- 172.21.15.53:0/2070524358 wait complete. 2023-11-28T11:22:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 33 2023-11-28T11:22:00.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:00.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.540+0000 7fb1791ff700 1 Processor -- start 2023-11-28T11:22:00.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.540+0000 7fb1791ff700 1 -- start start 2023-11-28T11:22:00.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb1791ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1740cd7c0 0x7fb1740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb1791ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740d1950 con 0x7fb174161340 2023-11-28T11:22:00.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb1791ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740ce140 con 0x7fb17415c9b0 2023-11-28T11:22:00.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb1791ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740b5800 con 0x7fb1740cd7c0 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb17259c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb174161340 0x7fb1740d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40662/0 (socket says 172.21.15.53:40662) 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb17259c700 1 -- 172.21.15.53:0/4195092713 learned_addr learned my addr 172.21.15.53:0/4195092713 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.541+0000 7fb17359e700 1 --2- 172.21.15.53:0/4195092713 >> v2:172.21.15.157:3300/0 conn(0x7fb1740cd7c0 0x7fb1740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143349476 0 0) 0x7fb1740d1950 con 0x7fb174161340 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c0035b0 con 0x7fb174161340 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882872809 0 0) 0x7fb1740ce140 con 0x7fb17415c9b0 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1740d1950 con 0x7fb17415c9b0 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177001680 0 0) 0x7fb15c0035b0 con 0x7fb174161340 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 >> v2:172.21.15.157:3300/0 conn(0x7fb1740cd7c0 msgr2=0x7fb1740cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 --2- 172.21.15.53:0/4195092713 >> v2:172.21.15.157:3300/0 conn(0x7fb1740cd7c0 0x7fb1740cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 >> v1:172.21.15.157:6789/0 conn(0x7fb17415c9b0 legacy=0x7fb1740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740d24f0 con 0x7fb174161340 2023-11-28T11:22:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fb168001670 con 0x7fb174161340 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 >> v1:172.21.15.53:6789/0 conn(0x7fb174161340 legacy=0x7fb1740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c003be0 0x7fb15c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 0x7fb15c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15c005250 0x7fb15c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740b5800 con 0x7fb15c004850 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c00c160 con 0x7fb15c003be0 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c00c320 con 0x7fb15c005250 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.542+0000 7fb17259c700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 0x7fb15c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb172d9d700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c003be0 0x7fb15c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17359e700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15c005250 0x7fb15c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 -- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15c005250 msgr2=0x7fb15c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15c005250 0x7fb15c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 -- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c003be0 msgr2=0x7fb15c003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c003be0 0x7fb15c003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 -- 172.21.15.53:0/4195092713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0035b0 con 0x7fb15c004850 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17359e700 1 -- 172.21.15.53:0/4195092713 reap_dead start 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb17259c700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 0x7fb15c004cc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c0e0 tx=0x7fb15c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465db4214a444350 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.543+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800cc80 con 0x7fb15c004850 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800cde0 con 0x7fb15c004850 2023-11-28T11:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb171d9b700 1 -- 172.21.15.53:0/4195092713 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168005ba0 con 0x7fb15c004850 2023-11-28T11:22:00.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 -- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 msgr2=0x7fb15c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 0x7fb15c004cc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c0e0 tx=0x7fb15c005880 comp rx=0 tx=0).stop 2023-11-28T11:22:00.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 -- 172.21.15.53:0/4195092713 shutdown_connections 2023-11-28T11:22:00.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 --2- 172.21.15.53:0/4195092713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb15c004850 0x7fb15c004cc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 -- 172.21.15.53:0/4195092713 >> 172.21.15.53:0/4195092713 conn(0x7fb174072600 msgr2=0x7fb1740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 -- 172.21.15.53:0/4195092713 shutdown_connections 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.544+0000 7fb1791ff700 1 -- 172.21.15.53:0/4195092713 wait complete. 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.546+0000 7fb1791ff700 1 Processor -- start 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.546+0000 7fb1791ff700 1 -- start start 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 0x7fb17414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cd260 0x7fb17415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740c8f30 con 0x7fb1741f2bc0 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17414daf0 con 0x7fb15c004850 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb1791ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740c82b0 con 0x7fb1740cd260 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb17259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb17259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50072/0 (socket says 172.21.15.53:50072) 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.547+0000 7fb17259c700 1 -- 172.21.15.53:0/2146893588 learned_addr learned my addr 172.21.15.53:0/2146893588 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17359e700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cd260 0x7fb17415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb172d9d700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 0x7fb17414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 -- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cd260 msgr2=0x7fb17415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cd260 0x7fb17415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 -- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 msgr2=0x7fb17414d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 0x7fb17414d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 -- 172.21.15.53:0/2146893588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740d1300 con 0x7fb1741f2bc0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb172d9d700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 0x7fb17414d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb17259c700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c130 tx=0x7fb168001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168004c80 con 0x7fb1741f2bc0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb168004e00 con 0x7fb1741f2bc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168007310 con 0x7fb1741f2bc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.548+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1740b5520 con 0x7fb1741f2bc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.549+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1740d15e0 con 0x7fb1741f2bc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.550+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1740c9070 con 0x7fb1741f2bc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.551+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb168010070 con 0x7fb1741f2bc0 2023-11-28T11:22:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.552+0000 7fb15b7fe700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.552+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb168086950 con 0x7fb1741f2bc0 2023-11-28T11:22:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.552+0000 7fb172d9d700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.553+0000 7fb172d9d700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 0x7fb16006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb164007fd0 tx=0x7fb164009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.556+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1740c9070 con 0x7fb1741f2bc0 2023-11-28T11:22:00.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.761+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1740c9070 con 0x7fb1741f2bc0 2023-11-28T11:22:00.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.762+0000 7fb15b7fe700 1 -- 172.21.15.53:0/2146893588 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18159 (secure 0 0 0) 0x7fb1740c9070 con 0x7fb1741f2bc0 2023-11-28T11:22:00.763 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:00.764 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:21:38.857031+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:46.350181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:47.505564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.7","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.764+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 msgr2=0x7fb16006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 0x7fb16006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb164007fd0 tx=0x7fb164009040 comp rx=0 tx=0).stop 2023-11-28T11:22:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 msgr2=0x7fb1740ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c130 tx=0x7fb168001930 comp rx=0 tx=0).stop 2023-11-28T11:22:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 shutdown_connections 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cd260 0x7fb17415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c004850 0x7fb17414d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 --2- 172.21.15.53:0/2146893588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1741f2bc0 0x7fb1740ccd20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 >> 172.21.15.53:0/2146893588 conn(0x7fb174072600 msgr2=0x7fb1740cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 shutdown_connections 2023-11-28T11:22:00.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.765+0000 7fb1791ff700 1 -- 172.21.15.53:0/2146893588 wait complete. 2023-11-28T11:22:00.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:22:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.987+0000 7f55e11ff700 1 Processor -- start 2023-11-28T11:22:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.987+0000 7f55e11ff700 1 -- start start 2023-11-28T11:22:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55e11ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55dc0d0ff0 0x7f55dc0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55e11ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55dc0b4750 0x7f55dc0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55e11ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc0cf220 con 0x7f55dc0d13c0 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55e11ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0b3e50 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55e11ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0cf470 con 0x7f55dc0b4750 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55dad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55dc0d0ff0 0x7f55dc0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55dad9d700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55dc0d0ff0 0x7f55dc0cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58746/0 (socket says 172.21.15.53:58746) 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.988+0000 7f55dad9d700 1 -- 172.21.15.53:0/4233909158 learned_addr learned my addr 172.21.15.53:0/4233909158 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55da59c700 1 --2- 172.21.15.53:0/4233909158 >> v2:172.21.15.157:3300/0 conn(0x7f55dc0b4750 0x7f55dc0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55dad9d700 1 -- 172.21.15.53:0/4233909158 >> v2:172.21.15.157:3300/0 conn(0x7f55dc0b4750 msgr2=0x7f55dc0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55dad9d700 1 --2- 172.21.15.53:0/4233909158 >> v2:172.21.15.157:3300/0 conn(0x7f55dc0b4750 0x7f55dc0b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55dad9d700 1 -- 172.21.15.53:0/4233909158 >> v1:172.21.15.157:6789/0 conn(0x7f55dc0d13c0 legacy=0x7f55dc0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55dad9d700 1 -- 172.21.15.53:0/4233909158 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0cf750 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55dad9d700 1 --2- 172.21.15.53:0/4233909158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0d0ff0 0x7f55dc0cff50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f55dc08d340 tx=0x7f55cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0936e91bc35d0fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55d9d9b700 1 -- 172.21.15.53:0/4233909158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc005e30 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55d9d9b700 1 -- 172.21.15.53:0/4233909158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc004030 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.989+0000 7f55d9d9b700 1 -- 172.21.15.53:0/4233909158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0043e0 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 -- 172.21.15.53:0/4233909158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0d0ff0 msgr2=0x7f55dc0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 --2- 172.21.15.53:0/4233909158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0d0ff0 0x7f55dc0cff50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f55dc08d340 tx=0x7f55cc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 -- 172.21.15.53:0/4233909158 shutdown_connections 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 --2- 172.21.15.53:0/4233909158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0d0ff0 0x7f55dc0cff50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 --2- 172.21.15.53:0/4233909158 >> v2:172.21.15.157:3300/0 conn(0x7f55dc0b4750 0x7f55dc0b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 -- 172.21.15.53:0/4233909158 >> 172.21.15.53:0/4233909158 conn(0x7f55dc09c3c0 msgr2=0x7f55dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 -- 172.21.15.53:0/4233909158 shutdown_connections 2023-11-28T11:22:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.990+0000 7f55e11ff700 1 -- 172.21.15.53:0/4233909158 wait complete. 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.992+0000 7f55e11ff700 1 Processor -- start 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.992+0000 7f55e11ff700 1 -- start start 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55dc0d0ff0 0x7f55dc0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55dc0d13c0 0x7f55dc1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc147440 con 0x7f55dc0b4750 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c9eb0 con 0x7f55dc0d13c0 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55e11ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0ce0a0 con 0x7f55dc0d0ff0 2023-11-28T11:22:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58752/0 (socket says 172.21.15.53:58752) 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 -- 172.21.15.53:0/827654572 learned_addr learned my addr 172.21.15.53:0/827654572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55db59e700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55dc0d13c0 0x7f55dc1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55da59c700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55dc0d0ff0 0x7f55dc0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 -- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55dc0d0ff0 msgr2=0x7f55dc0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55dc0d0ff0 0x7f55dc0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 -- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55dc0d13c0 msgr2=0x7f55dc1494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55dc0d13c0 0x7f55dc1494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.993+0000 7f55dad9d700 1 -- 172.21.15.53:0/827654572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc009500 con 0x7f55dc0b4750 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55dad9d700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009130 tx=0x7f55cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc003dd0 con 0x7f55dc0b4750 2023-11-28T11:22:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc011420 con 0x7f55dc0b4750 2023-11-28T11:22:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0d0a50 con 0x7f55dc0b4750 2023-11-28T11:22:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0117d0 con 0x7f55dc0b4750 2023-11-28T11:22:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.994+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc0cfa30 con 0x7f55dc0b4750 2023-11-28T11:22:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.995+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc0d0490 con 0x7f55dc0b4750 2023-11-28T11:22:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.996+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f55cc02c070 con 0x7f55dc0b4750 2023-11-28T11:22:00.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.997+0000 7f55c37fe700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 0x7f55c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.997+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f55cc086e20 con 0x7f55dc0b4750 2023-11-28T11:22:00.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.998+0000 7f55da59c700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 0x7f55c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:00.998+0000 7f55da59c700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 0x7f55c806a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0caaf0 tx=0x7f55d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.001+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55cc0535f0 con 0x7f55dc0b4750 2023-11-28T11:22:01.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.249+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55dc0d1e30 con 0x7f55dc0b4750 2023-11-28T11:22:01.880 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-11-28T11:22:01.879+0000 7fa6ddbf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.886+0000 7f55c37fe700 1 -- 172.21.15.53:0/827654572 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55cc056c10 con 0x7f55dc0b4750 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 msgr2=0x7f55c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 0x7f55c806a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0caaf0 tx=0x7f55d0008040 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 msgr2=0x7f55dc0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009130 tx=0x7f55cc004750 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 shutdown_connections 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f55c80682f0 0x7f55c806a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55dc0d13c0 0x7f55dc1494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0b4750 0x7f55dc0c9970 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 --2- 172.21.15.53:0/827654572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55dc0d0ff0 0x7f55dc0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 >> 172.21.15.53:0/827654572 conn(0x7f55dc09c3c0 msgr2=0x7f55dc06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:01.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 shutdown_connections 2023-11-28T11:22:01.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:01.889+0000 7f55e11ff700 1 -- 172.21.15.53:0/827654572 wait complete. 2023-11-28T11:22:01.892 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:22:01.901 INFO:tasks.ceph.mds.a.smithi053.stderr: -7366> 2023-11-28T11:20:52.979+0000 7fa7363f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:22:01.907 INFO:tasks.ceph.mds.a.smithi053.stderr: -3806> 2023-11-28T11:20:58.529+0000 7fa732bea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6135=0-4194304@1} caps={6135=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6135 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x558093d31600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:22:01.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:22:01.962 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-11-28T11:22:02.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.125+0000 7ff24abff700 1 Processor -- start 2023-11-28T11:22:02.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.125+0000 7ff24abff700 1 -- start start 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff24abff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff24415c9b0 0x7ff2440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff24abff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff244161340 0x7ff2440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff24abff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440d1950 con 0x7ff2440d2320 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff24abff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440b5800 con 0x7ff244161340 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff24abff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b5520 con 0x7ff24415c9b0 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.126+0000 7ff243fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff244161340 0x7ff2440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff244161340 0x7ff2440d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58776/0 (socket says 172.21.15.53:58776) 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 -- 172.21.15.53:0/1504291284 learned_addr learned my addr 172.21.15.53:0/1504291284 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff24899b700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff24415c9b0 0x7ff2440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 -- 172.21.15.53:0/1504291284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff24415c9b0 msgr2=0x7ff2440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff24415c9b0 0x7ff2440b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 -- 172.21.15.53:0/1504291284 >> v1:172.21.15.157:6789/0 conn(0x7ff2440d2320 legacy=0x7ff2440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 -- 172.21.15.53:0/1504291284 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440d1300 con 0x7ff244161340 2023-11-28T11:22:02.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.127+0000 7ff243fff700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff244161340 0x7ff2440d1de0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff2300098f0 tx=0x7ff230009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334d05cf3e416d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff2437fe700 1 -- 172.21.15.53:0/1504291284 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000b8a0 con 0x7ff244161340 2023-11-28T11:22:02.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff2437fe700 1 -- 172.21.15.53:0/1504291284 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000ba00 con 0x7ff244161340 2023-11-28T11:22:02.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff2437fe700 1 -- 172.21.15.53:0/1504291284 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000bd90 con 0x7ff244161340 2023-11-28T11:22:02.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 -- 172.21.15.53:0/1504291284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff244161340 msgr2=0x7ff2440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff244161340 0x7ff2440d1de0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff2300098f0 tx=0x7ff230009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 -- 172.21.15.53:0/1504291284 shutdown_connections 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff24415c9b0 0x7ff2440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 --2- 172.21.15.53:0/1504291284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff244161340 0x7ff2440d1de0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 -- 172.21.15.53:0/1504291284 >> 172.21.15.53:0/1504291284 conn(0x7ff244072600 msgr2=0x7ff2440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 -- 172.21.15.53:0/1504291284 shutdown_connections 2023-11-28T11:22:02.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.128+0000 7ff24abff700 1 -- 172.21.15.53:0/1504291284 wait complete. 2023-11-28T11:22:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.130+0000 7ff24abff700 1 Processor -- start 2023-11-28T11:22:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.130+0000 7ff24abff700 1 -- start start 2023-11-28T11:22:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2440d2320 0x7ff2440c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 0x7ff2440b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440c9310 con 0x7ff24415c9b0 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440c6410 con 0x7ff244161340 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24abff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440c89e0 con 0x7ff2440d2320 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff243fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff243fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58784/0 (socket says 172.21.15.53:58784) 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff243fff700 1 -- 172.21.15.53:0/2169692255 learned_addr learned my addr 172.21.15.53:0/2169692255 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24899b700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2440d2320 0x7ff2440c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.131+0000 7ff24919c700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 0x7ff2440b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 -- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2440d2320 msgr2=0x7ff2440c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2440d2320 0x7ff2440c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 -- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 msgr2=0x7ff2440b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 0x7ff2440b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 -- 172.21.15.53:0/2169692255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009550 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff24919c700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 0x7ff2440b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff243fff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff230000c00 tx=0x7ff230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230004380 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23001f070 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440b5130 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300036a0 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.132+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440d15e0 con 0x7ff24415c9b0 2023-11-28T11:22:02.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.134+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2440b52c0 con 0x7ff24415c9b0 2023-11-28T11:22:02.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.134+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff23002a020 con 0x7ff24415c9b0 2023-11-28T11:22:02.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.136+0000 7ff2417fa700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.136+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff2300873a0 con 0x7ff24415c9b0 2023-11-28T11:22:02.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.136+0000 7ff24899b700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.137+0000 7ff24899b700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 0x7ff22806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff238006b20 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.140+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2300539c0 con 0x7ff24415c9b0 2023-11-28T11:22:02.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.389+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff2440745c0 con 0x7ff24415c9b0 2023-11-28T11:22:02.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.887+0000 7ff2417fa700 1 -- 172.21.15.53:0/2169692255 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff230056fe0 con 0x7ff24415c9b0 2023-11-28T11:22:02.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 msgr2=0x7ff22806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 0x7ff22806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff238006b20 tx=0x7ff238008040 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 msgr2=0x7ff2440c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff230000c00 tx=0x7ff230004750 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 shutdown_connections 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2440d2320 0x7ff2440c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff244161340 0x7ff2440b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 --2- 172.21.15.53:0/2169692255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff24415c9b0 0x7ff2440c5ed0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 >> 172.21.15.53:0/2169692255 conn(0x7ff244072600 msgr2=0x7ff24415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 shutdown_connections 2023-11-28T11:22:02.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:02.890+0000 7ff24abff700 1 -- 172.21.15.53:0/2169692255 wait complete. 2023-11-28T11:22:02.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:22:03.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.223+0000 7f52819ff700 1 Processor -- start 2023-11-28T11:22:03.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.223+0000 7f52819ff700 1 -- start start 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f52819ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f527c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f52819ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f527c15dc30 0x7f527c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f52819ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c15e0a0 con 0x7f527c15a760 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f52819ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c15ab30 con 0x7f527c15dc30 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f52819ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c160d70 con 0x7f527c0cded0 2023-11-28T11:22:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f527affd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f527c15a760 0x7f527c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56982/0 (socket says 172.21.15.53:56982) 2023-11-28T11:22:03.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.224+0000 7f527affd700 1 -- 172.21.15.53:0/553217555 learned_addr learned my addr 172.21.15.53:0/553217555 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f527b7fe700 1 --2- 172.21.15.53:0/553217555 >> v2:172.21.15.157:3300/0 conn(0x7f527c15dc30 0x7f527c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f527a7fc700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f527c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003708212 0 0) 0x7f527c15e0a0 con 0x7f527c15a760 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52640035b0 con 0x7f527c15a760 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790197467 0 0) 0x7f52640035b0 con 0x7f527c15a760 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 msgr2=0x7f527c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f527c161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 >> v2:172.21.15.157:3300/0 conn(0x7f527c15dc30 msgr2=0x7f527c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 --2- 172.21.15.53:0/553217555 >> v2:172.21.15.157:3300/0 conn(0x7f527c15dc30 0x7f527c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0b5130 con 0x7f527c15a760 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f526c001670 con 0x7f527c15a760 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 >> v1:172.21.15.53:6789/0 conn(0x7f527c15a760 legacy=0x7f527c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264003a00 0x7f5264003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5264004850 0x7f5264004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f5264005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c15ab30 con 0x7f5264004850 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c160d70 con 0x7f5264003a00 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.225+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400ba40 con 0x7f527c0cded0 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527affd700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264003a00 0x7f5264003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5264004850 0x7f5264004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527b7fe700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f5264005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 -- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 msgr2=0x7f5264005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f527c0cded0 0x7f5264005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 -- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264003a00 msgr2=0x7f5264003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264003a00 0x7f5264003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 -- 172.21.15.53:0/553217555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640035b0 con 0x7f5264004850 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f527a7fc700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5264004850 0x7f5264004cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5270009b30 tx=0x7f5264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80e34e7d9307afd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000e6c0 con 0x7f5264004850 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527000e820 con 0x7f5264004850 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.226+0000 7f5279ffb700 1 -- 172.21.15.53:0/553217555 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000eb90 con 0x7f5264004850 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 -- 172.21.15.53:0/553217555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5264004850 msgr2=0x7f5264004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 --2- 172.21.15.53:0/553217555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5264004850 0x7f5264004cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5270009b30 tx=0x7f5264005880 comp rx=0 tx=0).stop 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f527b7fe700 1 -- 172.21.15.53:0/553217555 reap_dead start 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 -- 172.21.15.53:0/553217555 shutdown_connections 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 -- 172.21.15.53:0/553217555 >> 172.21.15.53:0/553217555 conn(0x7f527c072620 msgr2=0x7f527c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 -- 172.21.15.53:0/553217555 shutdown_connections 2023-11-28T11:22:03.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.227+0000 7f52819ff700 1 -- 172.21.15.53:0/553217555 wait complete. 2023-11-28T11:22:03.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.229+0000 7f52819ff700 1 Processor -- start 2023-11-28T11:22:03.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.229+0000 7f52819ff700 1 -- start start 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264004850 0x7f527c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 0x7f527c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0b8650 con 0x7f527c0c5f40 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0b7a40 con 0x7f527c0cc300 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f52819ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0b7d20 con 0x7f5264004850 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f527a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f527a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58804/0 (socket says 172.21.15.53:58804) 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f527a7fc700 1 -- 172.21.15.53:0/2912184498 learned_addr learned my addr 172.21.15.53:0/2912184498 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f527affd700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264004850 0x7f527c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.230+0000 7f527b7fe700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 0x7f527c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 -- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264004850 msgr2=0x7f527c0b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264004850 0x7f527c0b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 -- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 msgr2=0x7f527c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 0x7f527c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 -- 172.21.15.53:0/2912184498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0b5670 con 0x7f527c0c5f40 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527b7fe700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 0x7f527c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f527a7fc700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5264004760 tx=0x7f52700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270021070 con 0x7f527c0c5f40 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270004520 con 0x7f527c0c5f40 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c0b5390 con 0x7f527c0c5f40 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000e6c0 con 0x7f527c0c5f40 2023-11-28T11:22:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.231+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c160720 con 0x7f527c0c5f40 2023-11-28T11:22:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.232+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c0b8800 con 0x7f527c0c5f40 2023-11-28T11:22:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.233+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f527002e030 con 0x7f527c0c5f40 2023-11-28T11:22:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.234+0000 7f5263fff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 0x7f526806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.234+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f5270086fa0 con 0x7f527c0c5f40 2023-11-28T11:22:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.235+0000 7f527affd700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 0x7f526806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.235+0000 7f527affd700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 0x7f526806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f526c000fb0 tx=0x7f526c001e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.239+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f527c0b8800 con 0x7f527c0c5f40 2023-11-28T11:22:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.440+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f527c04e620 con 0x7f527c0c5f40 2023-11-28T11:22:03.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.894+0000 7f5263fff700 1 -- 172.21.15.53:0/2912184498 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5270053620 con 0x7f527c0c5f40 2023-11-28T11:22:03.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 msgr2=0x7f526806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 0x7f526806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f526c000fb0 tx=0x7f526c001e00 comp rx=0 tx=0).stop 2023-11-28T11:22:03.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 msgr2=0x7f527c0cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5264004760 tx=0x7f52700047c0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 shutdown_connections 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f52680682f0 0x7f526806a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264004850 0x7f527c0b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f527c0cc300 0x7f527c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 --2- 172.21.15.53:0/2912184498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f527c0c5f40 0x7f527c0cbdc0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.898+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 >> 172.21.15.53:0/2912184498 conn(0x7f527c072620 msgr2=0x7f527c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.899+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 shutdown_connections 2023-11-28T11:22:03.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:03.899+0000 7f52819ff700 1 -- 172.21.15.53:0/2912184498 wait complete. 2023-11-28T11:22:03.901 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:22:03.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:22:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.128+0000 7fdd36dff700 1 Processor -- start 2023-11-28T11:22:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.128+0000 7fdd36dff700 1 -- start start 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd36dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd300cded0 0x7fdd30161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd36dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd30160d70 con 0x7fdd3015a760 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd36dff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd300b5390 con 0x7fdd3015dc30 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd36dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3015e0d0 con 0x7fdd300cded0 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd34b9b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdd3015a760 0x7fdd300cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56992/0 (socket says 172.21.15.53:56992) 2023-11-28T11:22:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd34b9b700 1 -- 172.21.15.53:0/3569596423 learned_addr learned my addr 172.21.15.53:0/3569596423 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd2ffff700 1 --2- 172.21.15.53:0/3569596423 >> v2:172.21.15.157:3300/0 conn(0x7fdd300cded0 0x7fdd30161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724499302 0 0) 0x7fdd30160d70 con 0x7fdd3015a760 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.130+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd200035b0 con 0x7fdd3015a760 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1829110584 0 0) 0x7fdd200035b0 con 0x7fdd3015a760 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 >> v2:172.21.15.157:3300/0 conn(0x7fdd300cded0 msgr2=0x7fdd30161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 --2- 172.21.15.53:0/3569596423 >> v2:172.21.15.157:3300/0 conn(0x7fdd300cded0 0x7fdd30161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 >> v1:172.21.15.157:6789/0 conn(0x7fdd3015dc30 legacy=0x7fdd300d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300b5670 con 0x7fdd3015a760 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7fdd24001670 con 0x7fdd3015a760 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 >> v1:172.21.15.53:6789/0 conn(0x7fdd3015a760 legacy=0x7fdd300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd20003a50 0x7fdd20003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd20004850 0x7fdd20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3015e0d0 con 0x7fdd20005250 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2000c160 con 0x7fdd20003a50 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2000c320 con 0x7fdd20004850 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd3539c700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd34b9b700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd20003a50 0x7fdd20003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.131+0000 7fdd2ffff700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd20004850 0x7fdd20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 -- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd20004850 msgr2=0x7fdd20004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd20004850 0x7fdd20004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 -- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd20003a50 msgr2=0x7fdd20003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd20003a50 0x7fdd20003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 -- 172.21.15.53:0/3569596423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200035b0 con 0x7fdd20005250 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 -- 172.21.15.53:0/3569596423 reap_dead start 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd3539c700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd2000ba10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800c190 tx=0x7fdd20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7884568691429b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28001db0 con 0x7fdd20005250 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2800ccd0 con 0x7fdd20005250 2023-11-28T11:22:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.132+0000 7fdd2f7fe700 1 -- 172.21.15.53:0/3569596423 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28003c40 con 0x7fdd20005250 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 -- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 msgr2=0x7fdd2000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd2000ba10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800c190 tx=0x7fdd20005880 comp rx=0 tx=0).stop 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 -- 172.21.15.53:0/3569596423 shutdown_connections 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 --2- 172.21.15.53:0/3569596423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd2000ba10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 -- 172.21.15.53:0/3569596423 >> 172.21.15.53:0/3569596423 conn(0x7fdd30072620 msgr2=0x7fdd300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 -- 172.21.15.53:0/3569596423 shutdown_connections 2023-11-28T11:22:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.133+0000 7fdd36dff700 1 -- 172.21.15.53:0/3569596423 wait complete. 2023-11-28T11:22:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.135+0000 7fdd36dff700 1 Processor -- start 2023-11-28T11:22:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.135+0000 7fdd36dff700 1 -- start start 2023-11-28T11:22:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cd710 0x7fdd300b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300c8b90 0x7fdd300c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301505a0 con 0x7fdd20005250 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300c9540 con 0x7fdd300cd710 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd36dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300b7200 con 0x7fdd300c8b90 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd34b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd34b9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58834/0 (socket says 172.21.15.53:58834) 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd34b9b700 1 -- 172.21.15.53:0/2075677091 learned_addr learned my addr 172.21.15.53:0/2075677091 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd2ffff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cd710 0x7fdd300b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.136+0000 7fdd3539c700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300c8b90 0x7fdd300c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 -- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300c8b90 msgr2=0x7fdd300c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300c8b90 0x7fdd300c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 -- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cd710 msgr2=0x7fdd300b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cd710 0x7fdd300b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 -- 172.21.15.53:0/2075677091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30160a00 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd34b9b700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdd24007fd0 tx=0x7fdd24007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24012560 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd240126e0 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd30160720 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24012a90 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.137+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd30161200 con 0x7fdd20005250 2023-11-28T11:22:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.138+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd30150300 con 0x7fdd20005250 2023-11-28T11:22:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.139+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd24012bf0 con 0x7fdd20005250 2023-11-28T11:22:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.140+0000 7fdd2d7fa700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 0x7fdd1406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.140+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5826+0+0 (secure 0 0 0) 0x7fdd24085e40 con 0x7fdd20005250 2023-11-28T11:22:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.141+0000 7fdd2ffff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 0x7fdd1406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.141+0000 7fdd2ffff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 0x7fdd1406a7d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdd3015ae70 tx=0x7fdd1c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.144+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd30150300 con 0x7fdd20005250 2023-11-28T11:22:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.346+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdd301506e0 con 0x7fdd20005250 2023-11-28T11:22:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.901+0000 7fdd2d7fa700 1 -- 172.21.15.53:0/2075677091 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdd301506e0 con 0x7fdd20005250 2023-11-28T11:22:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 msgr2=0x7fdd1406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 0x7fdd1406a7d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdd3015ae70 tx=0x7fdd1c009210 comp rx=0 tx=0).stop 2023-11-28T11:22:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 msgr2=0x7fdd300b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdd24007fd0 tx=0x7fdd24007570 comp rx=0 tx=0).stop 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 shutdown_connections 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fdd14068320 0x7fdd1406a7d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd300c8b90 0x7fdd300c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd20005250 0x7fdd300b8a40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 --2- 172.21.15.53:0/2075677091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd300cd710 0x7fdd300b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 >> 172.21.15.53:0/2075677091 conn(0x7fdd30072620 msgr2=0x7fdd300d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 shutdown_connections 2023-11-28T11:22:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:04.906+0000 7fdd36dff700 1 -- 172.21.15.53:0/2075677091 wait complete. 2023-11-28T11:22:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-11-28T11:22:04.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:22:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.135+0000 7f6e78bff700 1 Processor -- start 2023-11-28T11:22:05.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.135+0000 7f6e78bff700 1 -- start start 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.136+0000 7f6e78bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7415c9b0 0x7f6e740b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e78bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e740cd7c0 0x7f6e740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e78bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e740d1950 con 0x7f6e74161340 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e78bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e740b5800 con 0x7f6e7415c9b0 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e78bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e740ce140 con 0x7f6e740cd7c0 2023-11-28T11:22:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7415c9b0 0x7f6e740b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7415c9b0 0x7f6e740b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58842/0 (socket says 172.21.15.53:58842) 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e6bfff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6e74161340 0x7f6e740d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56994/0 (socket says 172.21.15.53:56994) 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 -- 172.21.15.53:0/2879900865 learned_addr learned my addr 172.21.15.53:0/2879900865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e72d9d700 1 --2- 172.21.15.53:0/2879900865 >> v2:172.21.15.157:3300/0 conn(0x7f6e740cd7c0 0x7f6e740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 -- 172.21.15.53:0/2879900865 >> v2:172.21.15.157:3300/0 conn(0x7f6e740cd7c0 msgr2=0x7f6e740cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 --2- 172.21.15.53:0/2879900865 >> v2:172.21.15.157:3300/0 conn(0x7f6e740cd7c0 0x7f6e740cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 -- 172.21.15.53:0/2879900865 >> v1:172.21.15.53:6789/0 conn(0x7f6e74161340 legacy=0x7f6e740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e7259c700 1 -- 172.21.15.53:0/2879900865 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e740d24f0 con 0x7f6e7415c9b0 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.137+0000 7f6e71d9b700 1 -- 172.21.15.53:0/2879900865 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646804445 0 0) 0x7f6e740d1950 con 0x7f6e74161340 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e7259c700 1 --2- 172.21.15.53:0/2879900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7415c9b0 0x7f6e740b4bf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6e7406c170 tx=0x7f6e64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1f4ad3ae85080 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e71d9b700 1 -- 172.21.15.53:0/2879900865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400b870 con 0x7f6e7415c9b0 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e71d9b700 1 -- 172.21.15.53:0/2879900865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400b9d0 con 0x7f6e7415c9b0 2023-11-28T11:22:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e71d9b700 1 -- 172.21.15.53:0/2879900865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400bd60 con 0x7f6e7415c9b0 2023-11-28T11:22:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e78bff700 1 -- 172.21.15.53:0/2879900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7415c9b0 msgr2=0x7f6e740b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.138+0000 7f6e78bff700 1 --2- 172.21.15.53:0/2879900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7415c9b0 0x7f6e740b4bf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6e7406c170 tx=0x7f6e64009bd0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 -- 172.21.15.53:0/2879900865 shutdown_connections 2023-11-28T11:22:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 --2- 172.21.15.53:0/2879900865 >> v2:172.21.15.157:3300/0 conn(0x7f6e740cd7c0 0x7f6e740cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 --2- 172.21.15.53:0/2879900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7415c9b0 0x7f6e740b4bf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 -- 172.21.15.53:0/2879900865 >> 172.21.15.53:0/2879900865 conn(0x7f6e74072600 msgr2=0x7f6e740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 -- 172.21.15.53:0/2879900865 shutdown_connections 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.139+0000 7f6e78bff700 1 -- 172.21.15.53:0/2879900865 wait complete. 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.140+0000 7f6e78bff700 1 Processor -- start 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.140+0000 7f6e78bff700 1 -- start start 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 0x7f6e741e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e74161340 0x7f6e741e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7415d350 con 0x7f6e740cd7c0 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740c7750 con 0x7f6e7415c9b0 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.141+0000 7f6e78bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7415d5a0 con 0x7f6e74161340 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58858/0 (socket says 172.21.15.53:58858) 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 -- 172.21.15.53:0/3123403204 learned_addr learned my addr 172.21.15.53:0/3123403204 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e6bfff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 0x7f6e741e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e72d9d700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e74161340 0x7f6e741e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 -- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e74161340 msgr2=0x7f6e741e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e74161340 0x7f6e741e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 -- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 msgr2=0x7f6e741e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 0x7f6e741e8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 -- 172.21.15.53:0/3123403204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64009550 con 0x7f6e740cd7c0 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e6bfff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 0x7f6e741e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.142+0000 7f6e7259c700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.143+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64004520 con 0x7f6e740cd7c0 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.143+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6401e070 con 0x7f6e740cd7c0 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.143+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640038a0 con 0x7f6e740cd7c0 2023-11-28T11:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.143+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e740d1300 con 0x7f6e740cd7c0 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.143+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e740b5520 con 0x7f6e740cd7c0 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.144+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e740d15e0 con 0x7f6e740cd7c0 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.145+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e6400f050 con 0x7f6e740cd7c0 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.146+0000 7f6e6affd700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 0x7f6e5406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.146+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6e64020070 con 0x7f6e740cd7c0 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.146+0000 7f6e6bfff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 0x7f6e5406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.147+0000 7f6e6bfff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 0x7f6e5406a770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c000f60 tx=0x7f6e5c007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.150+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e64053a70 con 0x7f6e740cd7c0 2023-11-28T11:22:05.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.361+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6e740d2af0 con 0x7f6e740cd7c0 2023-11-28T11:22:05.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.365+0000 7f6e6affd700 1 -- 172.21.15.53:0/3123403204 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6e64057090 con 0x7f6e740cd7c0 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 msgr2=0x7f6e5406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 0x7f6e5406a770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c000f60 tx=0x7f6e5c007d30 comp rx=0 tx=0).stop 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 msgr2=0x7f6e74143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64004750 comp rx=0 tx=0).stop 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 shutdown_connections 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f6e540682c0 0x7f6e5406a770 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e74161340 0x7f6e741e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e740cd7c0 0x7f6e74143200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 --2- 172.21.15.53:0/3123403204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7415c9b0 0x7f6e741e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 >> 172.21.15.53:0/3123403204 conn(0x7f6e74072600 msgr2=0x7f6e7406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 shutdown_connections 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.368+0000 7f6e78bff700 1 -- 172.21.15.53:0/3123403204 wait complete. 2023-11-28T11:22:05.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:22:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.589+0000 7fea557ff700 1 Processor -- start 2023-11-28T11:22:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.589+0000 7fea557ff700 1 -- start start 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea557ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fea50161340 0x7fea500d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea557ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea500d2320 0x7fea500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea557ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea500d1950 con 0x7fea5015c9b0 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea557ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea500b5800 con 0x7fea50161340 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea557ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500b5300 con 0x7fea500d2320 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4e7fc700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fea50161340 0x7fea500d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4effd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fea5015c9b0 0x7fea500b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57010/0 (socket says 172.21.15.53:57010) 2023-11-28T11:22:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4effd700 1 -- 172.21.15.53:0/3442950139 learned_addr learned my addr 172.21.15.53:0/3442950139 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4f7fe700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea500d2320 0x7fea500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4e7fc700 1 -- 172.21.15.53:0/3442950139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea500d2320 msgr2=0x7fea500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4e7fc700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea500d2320 0x7fea500cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4e7fc700 1 -- 172.21.15.53:0/3442950139 >> v1:172.21.15.53:6789/0 conn(0x7fea5015c9b0 legacy=0x7fea500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4e7fc700 1 -- 172.21.15.53:0/3442950139 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500b5520 con 0x7fea50161340 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.591+0000 7fea4dffb700 1 -- 172.21.15.53:0/3442950139 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667068506 0 0) 0x7fea500d1950 con 0x7fea5015c9b0 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea4e7fc700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea50161340 0x7fea500d1de0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fea44009840 tx=0x7fea44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610791efebdf6980 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea4dffb700 1 -- 172.21.15.53:0/3442950139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011830 con 0x7fea50161340 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea4dffb700 1 -- 172.21.15.53:0/3442950139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44011990 con 0x7fea50161340 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea4dffb700 1 -- 172.21.15.53:0/3442950139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011d20 con 0x7fea50161340 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea557ff700 1 -- 172.21.15.53:0/3442950139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea50161340 msgr2=0x7fea500d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.592+0000 7fea557ff700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea50161340 0x7fea500d1de0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fea44009840 tx=0x7fea44009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 -- 172.21.15.53:0/3442950139 shutdown_connections 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea500d2320 0x7fea500cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 --2- 172.21.15.53:0/3442950139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea50161340 0x7fea500d1de0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 -- 172.21.15.53:0/3442950139 >> 172.21.15.53:0/3442950139 conn(0x7fea50072600 msgr2=0x7fea500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 -- 172.21.15.53:0/3442950139 shutdown_connections 2023-11-28T11:22:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.593+0000 7fea557ff700 1 -- 172.21.15.53:0/3442950139 wait complete. 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.594+0000 7fea557ff700 1 Processor -- start 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.595+0000 7fea557ff700 1 -- start start 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.595+0000 7fea557ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea557ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea5015c9b0 0x7fea50150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea557ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea50161340 0x7fea50159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea557ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea501e8320 con 0x7fea500d2320 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea557ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea501e76a0 con 0x7fea50161340 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea557ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea501e7980 con 0x7fea5015c9b0 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58868/0 (socket says 172.21.15.53:58868) 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 -- 172.21.15.53:0/28487605 learned_addr learned my addr 172.21.15.53:0/28487605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4e7fc700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea5015c9b0 0x7fea50150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4f7fe700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea50161340 0x7fea50159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 -- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea5015c9b0 msgr2=0x7fea50150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea5015c9b0 0x7fea50150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 -- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea50161340 msgr2=0x7fea50159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea50161340 0x7fea50159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 -- 172.21.15.53:0/28487605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440094a0 con 0x7fea500d2320 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.596+0000 7fea4effd700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c550 tx=0x7fea4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.597+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40004c80 con 0x7fea500d2320 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.597+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40004e00 con 0x7fea500d2320 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.597+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500d15e0 con 0x7fea500d2320 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.597+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40007310 con 0x7fea500d2320 2023-11-28T11:22:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.597+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea500d1300 con 0x7fea500d2320 2023-11-28T11:22:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.598+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5015cd80 con 0x7fea500d2320 2023-11-28T11:22:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.599+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea4001a020 con 0x7fea500d2320 2023-11-28T11:22:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.600+0000 7fea377fe700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 0x7fea3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.600+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fea40085690 con 0x7fea500d2320 2023-11-28T11:22:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.600+0000 7fea4e7fc700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 0x7fea3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.601+0000 7fea4e7fc700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 0x7fea3c06a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea440090b0 tx=0x7fea44008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.604+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea40052170 con 0x7fea500d2320 2023-11-28T11:22:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.891+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fea500745c0 con 0x7fea500d2320 2023-11-28T11:22:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.906+0000 7fea377fe700 1 -- 172.21.15.53:0/28487605 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v229) v1 ==== 169+0+0 (secure 0 0 0) 0x7fea40055790 con 0x7fea500d2320 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.908+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 msgr2=0x7fea3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.908+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 0x7fea3c06a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea440090b0 tx=0x7fea44008030 comp rx=0 tx=0).stop 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 msgr2=0x7fea501e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c550 tx=0x7fea4000c910 comp rx=0 tx=0).stop 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 shutdown_connections 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea3c0682f0 0x7fea3c06a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea50161340 0x7fea50159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea500d2320 0x7fea501e6e20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 --2- 172.21.15.53:0/28487605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea5015c9b0 0x7fea50150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 >> 172.21.15.53:0/28487605 conn(0x7fea50072600 msgr2=0x7fea5015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 shutdown_connections 2023-11-28T11:22:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:05.909+0000 7fea557ff700 1 -- 172.21.15.53:0/28487605 wait complete. 2023-11-28T11:22:05.929 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:22:05.929 INFO:tasks.cephfs_test_runner: 2023-11-28T11:22:05.929 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:22:05.929 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner: 2023-11-28T11:22:05.930 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:Ran 3 tests in 218.969s 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner: 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner: 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:22:05.931 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:22:05.932 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:22:05.932 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:22:05.932 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:22:05.932 INFO:tasks.cephfs_test_runner: 2023-11-28T11:22:05.932 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:22:05.998 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e54fae1fa2b74ba7a1845d8c80bfcb76 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:22:06.000 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:22:06.009 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T11:22:06.025 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T11:22:06.025 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:22:06.025 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:06.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:22:06.081 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-28T11:22:06.081 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:06.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351432k,nr_inodes=4087858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:22:06.164 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:22:06.164 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:06.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:22:06.216 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-28T11:22:06.216 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:06.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:22:06.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.301 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:22:06.344 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:22:06.344 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:22:06.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:22:06.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:22:06.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-28T11:22:06.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.655 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-28T11:22:06.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-28T11:22:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-11-28T11:22:06.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:06.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.782 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:22:06.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.905 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:22:06.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:06.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-28T11:22:07.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:07.095 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-28T11:22:07.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:07.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-28T11:22:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-11-28T11:22:07.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:07.257 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:22:07.267 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:22:07.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:07.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.474+0000 7f993b9ff700 1 Processor -- start 2023-11-28T11:22:07.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.475+0000 7f993b9ff700 1 -- start start 2023-11-28T11:22:07.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f993b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 0x7f99340b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f993b9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340ccc00 con 0x7f99340cdc80 2023-11-28T11:22:07.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f993b9ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340ccec0 con 0x7f9934159860 2023-11-28T11:22:07.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f993b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99340cd1a0 con 0x7f9934159490 2023-11-28T11:22:07.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f9939f9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f99340cdc80 0x7f993406fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57018/0 (socket says 172.21.15.53:57018) 2023-11-28T11:22:07.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f9939f9c700 1 -- 172.21.15.53:0/2914300060 learned_addr learned my addr 172.21.15.53:0/2914300060 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:07.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.476+0000 7f993979b700 1 --2- 172.21.15.53:0/2914300060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 0x7f99340b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112672809 0 0) 0x7f99340ccc00 con 0x7f99340cdc80 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99200057c0 con 0x7f99340cdc80 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316180460 0 0) 0x7f99340ccec0 con 0x7f9934159860 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f993979b700 1 -- 172.21.15.53:0/2914300060 >> v1:172.21.15.157:6789/0 conn(0x7f9934159860 legacy=0x7f99340c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f993979b700 1 -- 172.21.15.53:0/2914300060 >> v1:172.21.15.53:6789/0 conn(0x7f99340cdc80 legacy=0x7f993406fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f993979b700 1 -- 172.21.15.53:0/2914300060 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340cd480 con 0x7f9934159490 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f993979b700 1 --2- 172.21.15.53:0/2914300060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 0x7f99340b0670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9930009870 tx=0x7f9930009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9772ddd30eb97ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011830 con 0x7f9934159490 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.477+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930011990 con 0x7f9934159490 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f992bfff700 1 -- 172.21.15.53:0/2914300060 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011d20 con 0x7f9934159490 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 -- 172.21.15.53:0/2914300060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 msgr2=0x7f99340b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2914300060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 0x7f99340b0670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9930009870 tx=0x7f9930009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 -- 172.21.15.53:0/2914300060 shutdown_connections 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2914300060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9934159490 0x7f99340b0670 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 -- 172.21.15.53:0/2914300060 >> 172.21.15.53:0/2914300060 conn(0x7f99340cb670 msgr2=0x7f99340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 -- 172.21.15.53:0/2914300060 shutdown_connections 2023-11-28T11:22:07.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.478+0000 7f993b9ff700 1 -- 172.21.15.53:0/2914300060 wait complete. 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.480+0000 7f993b9ff700 1 Processor -- start 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.480+0000 7f993b9ff700 1 -- start start 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9934159490 0x7f9934153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99340b1a20 0x7f9934156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340b1fa0 con 0x7f99340cdc80 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340b2280 con 0x7f9934159490 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993b9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99340b2560 con 0x7f99340b1a20 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993979b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58872/0 (socket says 172.21.15.53:58872) 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f993979b700 1 -- 172.21.15.53:0/2916319933 learned_addr learned my addr 172.21.15.53:0/2916319933 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f9939f9c700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99340b1a20 0x7f9934156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.481+0000 7f9938f9a700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9934159490 0x7f9934153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 -- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99340b1a20 msgr2=0x7f9934156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99340b1a20 0x7f9934156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 -- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9934159490 msgr2=0x7f9934153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9934159490 0x7f9934153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 -- 172.21.15.53:0/2916319933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300094a0 con 0x7f99340cdc80 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993979b700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99300090b0 tx=0x7f9930004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993001a7e0 con 0x7f99340cdc80 2023-11-28T11:22:07.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993001a940 con 0x7f99340cdc80 2023-11-28T11:22:07.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993001acd0 con 0x7f99340cdc80 2023-11-28T11:22:07.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99340b0bb0 con 0x7f99340cdc80 2023-11-28T11:22:07.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.482+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99340cd760 con 0x7f99340cdc80 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.484+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99340b0d40 con 0x7f99340cdc80 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.484+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f993000f050 con 0x7f99340cdc80 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.485+0000 7f9929ffb700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 0x7f991806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.485+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9930086830 con 0x7f99340cdc80 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.486+0000 7f9938f9a700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 0x7f991806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.486+0000 7f9938f9a700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 0x7f991806a7d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f99341588b0 tx=0x7f9924009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.489+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99300532c0 con 0x7f99340cdc80 2023-11-28T11:22:07.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.692+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99340745c0 con 0x7f9918068320 2023-11-28T11:22:07.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.696+0000 7f9929ffb700 1 -- 172.21.15.53:0/2916319933 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73015 (secure 0 0 0) 0x7f99340745c0 con 0x7f9918068320 2023-11-28T11:22:07.701 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:07.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 msgr2=0x7f991806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 0x7f991806a7d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f99341588b0 tx=0x7f9924009040 comp rx=0 tx=0).stop 2023-11-28T11:22:07.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 msgr2=0x7f99340b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99300090b0 tx=0x7f9930004750 comp rx=0 tx=0).stop 2023-11-28T11:22:07.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 shutdown_connections 2023-11-28T11:22:07.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f9918068320 0x7f991806a7d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99340b1a20 0x7f9934156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99340cdc80 0x7f99340b3040 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 --2- 172.21.15.53:0/2916319933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9934159490 0x7f9934153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.698+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 >> 172.21.15.53:0/2916319933 conn(0x7f99340cb670 msgr2=0x7f99340ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.699+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 shutdown_connections 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.699+0000 7f993b9ff700 1 -- 172.21.15.53:0/2916319933 wait complete. 2023-11-28T11:22:07.705 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:22:07.718 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2023-11-28T11:22:06.998899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":113280,"kb_used_data":7360,"kb_used_omap":0,"kb_used_meta":105856,"kb_avail":754861440,"statfs":{"total":773094113280,"available":772978114560,"internally_reserved":0,"allocated":7536640,"data_stored":3991314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":108396544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-78,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001304"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.781077+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:22:04.781077+0000","last_peered":"2023-11-28T11:22:04.781077+0000","last_clean":"2023-11-28T11:22:04.781077+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:22:04.781077+0000","last_undegraded":"2023-11-28T11:22:04.781077+0000","last_fullsized":"2023-11-28T11:22:04.781077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.898460+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:22:03.898460+0000","last_peered":"2023-11-28T11:22:03.898460+0000","last_clean":"2023-11-28T11:22:03.898460+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:22:03.898460+0000","last_undegraded":"2023-11-28T11:22:03.898460+0000","last_fullsized":"2023-11-28T11:22:03.898460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:22:01.649406+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:22:01.649406+0000","last_peered":"2023-11-28T11:22:01.649406+0000","last_clean":"2023-11-28T11:22:01.649406+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:22:01.649406+0000","last_undegraded":"2023-11-28T11:22:01.649406+0000","last_fullsized":"2023-11-28T11:22:01.649406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.897644+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:22:03.897644+0000","last_peered":"2023-11-28T11:22:03.897644+0000","last_clean":"2023-11-28T11:22:03.897644+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:22:03.897644+0000","last_undegraded":"2023-11-28T11:22:03.897644+0000","last_fullsized":"2023-11-28T11:22:03.897644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424583+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:22:04.424583+0000","last_peered":"2023-11-28T11:22:04.424583+0000","last_clean":"2023-11-28T11:22:04.424583+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:22:04.424583+0000","last_undegraded":"2023-11-28T11:22:04.424583+0000","last_fullsized":"2023-11-28T11:22:04.424583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424494+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:22:04.424494+0000","last_peered":"2023-11-28T11:22:04.424494+0000","last_clean":"2023-11-28T11:22:04.424494+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:22:04.424494+0000","last_undegraded":"2023-11-28T11:22:04.424494+0000","last_fullsized":"2023-11-28T11:22:04.424494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.425950+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:22:04.425950+0000","last_peered":"2023-11-28T11:22:04.425950+0000","last_clean":"2023-11-28T11:22:04.425950+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:22:04.425950+0000","last_undegraded":"2023-11-28T11:22:04.425950+0000","last_fullsized":"2023-11-28T11:22:04.425950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.263216+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:22:05.263216+0000","last_peered":"2023-11-28T11:22:05.263216+0000","last_clean":"2023-11-28T11:22:05.263216+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:22:05.263216+0000","last_undegraded":"2023-11-28T11:22:05.263216+0000","last_fullsized":"2023-11-28T11:22:05.263216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.257573+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:22:05.257573+0000","last_peered":"2023-11-28T11:22:05.257573+0000","last_clean":"2023-11-28T11:22:05.257573+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:22:05.257573+0000","last_undegraded":"2023-11-28T11:22:05.257573+0000","last_fullsized":"2023-11-28T11:22:05.257573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15128,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356712,"statfs":{"total":96636764160,"available":96621273088,"internally_reserved":0,"allocated":1392640,"data_stored":943512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.87,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.996}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.204,"5min":2.23,"15min":2.23},"last":0.85699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.147,"15min":1.147},"last":0.78000000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.56499999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70699999999999996,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.625},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0269999999999999,"5min":1.627,"15min":1.627},"last":0.80800000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.876,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13904,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357936,"statfs":{"total":96636764160,"available":96622526464,"internally_reserved":0,"allocated":794624,"data_stored":352180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94999999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85299999999999998,"5min":1.181,"15min":1.181},"last":0.94699999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":1.774,"15min":1.774},"last":1.143}]},{"osd":6,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96699999999999997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.97399999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14932,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356908,"statfs":{"total":96636764160,"available":96621473792,"internally_reserved":0,"allocated":1388544,"data_stored":942493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":1.052,"15min":1.052},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.74399999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.503},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0249999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79100000000000004,"15min":0.80800000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2729999999999999,"5min":1.458,"15min":1.458},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.84199999999999997,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1639999999999999,"5min":1.591,"15min":1.591},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.64100000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.208,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.046,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.02}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13620,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358220,"statfs":{"total":96636764160,"available":96622817280,"internally_reserved":0,"allocated":765952,"data_stored":340553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.58099999999999996,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63200000000000001,"5min":1.04,"15min":1.04},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58299999999999996,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":4.093,"15min":4.093},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68200000000000005,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":3.956,"15min":3.956},"last":0.41699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76100000000000001,"15min":0.81899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0369999999999999,"5min":2.028,"15min":2.028},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.147,"15min":1.147},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.95199999999999996},"min":{"1min":0.58399999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.024,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0880000000000001,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.90800000000000003,"15min":1.0249999999999999},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.177,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80500000000000005,"15min":0.85499999999999998},"min":{"1min":0.65900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":1.272,"15min":1.272},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.86899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.56200000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14036,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357804,"statfs":{"total":96636764160,"available":96622391296,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48799999999999999,"15min":0.54300000000000004},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59999999999999998,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":3.004,"15min":3.004},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57399999999999995,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69599999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":3.274,"15min":3.274},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.034,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.89400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.93300000000000005},"min":{"1min":0.66700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0860000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.077,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.90700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":14100,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357740,"statfs":{"total":96636764160,"available":96622325760,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97699999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.746}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96899999999999997,"5min":1.847,"15min":1.847},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":1.615,"15min":1.615},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.099,"5min":1.127,"15min":1.127},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.083,"5min":1.089,"15min":1.089},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.127,"5min":1.296,"15min":1.296},"last":0.495},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.212,"15min":1.212},"last":0.52600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13908,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357932,"statfs":{"total":96636764160,"available":96622522368,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92900000000000005,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.72599999999999998},"min":{"1min":0.54800000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.67600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97699999999999998,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":2.661,"15min":2.661},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.73299999999999998},"min":{"1min":0.53700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86099999999999999,"5min":5.375,"15min":5.375},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.333,"15min":1.333},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:07.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:07.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.922+0000 7f2f1f3ff700 1 Processor -- start 2023-11-28T11:22:07.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.923+0000 7f2f1f3ff700 1 -- start start 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1f3ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f180cd7c0 0x7f2f180cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1f3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f180d1950 con 0x7f2f18161340 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1f3ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f180ce140 con 0x7f2f1815c9b0 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1f3ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f180b5800 con 0x7f2f180cd7c0 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1c99a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2f18161340 0x7f2f180d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57030/0 (socket says 172.21.15.53:57030) 2023-11-28T11:22:07.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1c99a700 1 -- 172.21.15.53:0/1268486063 learned_addr learned my addr 172.21.15.53:0/1268486063 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f1d99c700 1 --2- 172.21.15.53:0/1268486063 >> v2:172.21.15.157:3300/0 conn(0x7f2f180cd7c0 0x7f2f180cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443131374 0 0) 0x7f2f180d1950 con 0x7f2f18161340 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.924+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f040035b0 con 0x7f2f18161340 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563800321 0 0) 0x7f2f180ce140 con 0x7f2f1815c9b0 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f180d1950 con 0x7f2f1815c9b0 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f1d99c700 1 -- 172.21.15.53:0/1268486063 >> v1:172.21.15.157:6789/0 conn(0x7f2f1815c9b0 legacy=0x7f2f180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f1d99c700 1 -- 172.21.15.53:0/1268486063 >> v1:172.21.15.53:6789/0 conn(0x7f2f18161340 legacy=0x7f2f180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182477845 0 0) 0x7f2f040035b0 con 0x7f2f18161340 2023-11-28T11:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f1d99c700 1 -- 172.21.15.53:0/1268486063 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180d24f0 con 0x7f2f180cd7c0 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f1d99c700 1 --2- 172.21.15.53:0/1268486063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f180cd7c0 0x7f2f180cdc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009d90 tx=0x7f2f1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a556b0a52b2251 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000e070 con 0x7f2f180cd7c0 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f10003a20 con 0x7f2f180cd7c0 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.925+0000 7f2f0ffff700 1 -- 172.21.15.53:0/1268486063 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08001670 con 0x7f2f180cd7c0 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1268486063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f180cd7c0 msgr2=0x7f2f180cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1268486063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f180cd7c0 0x7f2f180cdc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009d90 tx=0x7f2f1000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1268486063 shutdown_connections 2023-11-28T11:22:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1268486063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f180cd7c0 0x7f2f180cdc00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1268486063 >> 172.21.15.53:0/1268486063 conn(0x7f2f18072600 msgr2=0x7f2f180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1268486063 shutdown_connections 2023-11-28T11:22:07.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.926+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1268486063 wait complete. 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.928+0000 7f2f1f3ff700 1 Processor -- start 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.928+0000 7f2f1f3ff700 1 -- start start 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f180cd7c0 0x7f2f18148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 0x7f2f181e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181466b0 con 0x7f2f1815c9b0 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181599f0 con 0x7f2f18161340 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1f3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f18148c60 con 0x7f2f180cd7c0 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1c99a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1c99a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58880/0 (socket says 172.21.15.53:58880) 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1c99a700 1 -- 172.21.15.53:0/1620383169 learned_addr learned my addr 172.21.15.53:0/1620383169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1d19b700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f180cd7c0 0x7f2f18148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.929+0000 7f2f1d99c700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 0x7f2f181e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 -- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f180cd7c0 msgr2=0x7f2f18148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f180cd7c0 0x7f2f18148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 -- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 msgr2=0x7f2f181e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 0x7f2f181e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 -- 172.21.15.53:0/1620383169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10009a70 con 0x7f2f1815c9b0 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1d99c700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 0x7f2f181e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1c99a700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c750 tx=0x7f2f0800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08004c80 con 0x7f2f1815c9b0 2023-11-28T11:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f08004de0 con 0x7f2f1815c9b0 2023-11-28T11:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f080072d0 con 0x7f2f1815c9b0 2023-11-28T11:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.930+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f180d1300 con 0x7f2f1815c9b0 2023-11-28T11:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.931+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f180b5520 con 0x7f2f1815c9b0 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.932+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f180d15e0 con 0x7f2f1815c9b0 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.932+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f0801a020 con 0x7f2f1815c9b0 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.933+0000 7f2f0dffb700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.933+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2f08085750 con 0x7f2f1815c9b0 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.934+0000 7f2f1d19b700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.934+0000 7f2f1d19b700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 0x7f2efc06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f140011e0 tx=0x7f2f14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:07.938+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f080521e0 con 0x7f2f1815c9b0 2023-11-28T11:22:08.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.136+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f180d0600 con 0x7f2efc0682f0 2023-11-28T11:22:08.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.140+0000 7f2f0dffb700 1 -- 172.21.15.53:0/1620383169 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73015 (secure 0 0 0) 0x7f2f180d0600 con 0x7f2efc0682f0 2023-11-28T11:22:08.141 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:08.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 msgr2=0x7f2efc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 0x7f2efc06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f140011e0 tx=0x7f2f14009040 comp rx=0 tx=0).stop 2023-11-28T11:22:08.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 msgr2=0x7f2f181e8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c750 tx=0x7f2f0800ca60 comp rx=0 tx=0).stop 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 shutdown_connections 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f180cd7c0 0x7f2f18148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18161340 0x7f2f181e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 --2- 172.21.15.53:0/1620383169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1815c9b0 0x7f2f181e8bf0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 >> 172.21.15.53:0/1620383169 conn(0x7f2f18072600 msgr2=0x7f2f180b8090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 shutdown_connections 2023-11-28T11:22:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.142+0000 7f2f1f3ff700 1 -- 172.21.15.53:0/1620383169 wait complete. 2023-11-28T11:22:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:22:08.161 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2023-11-28T11:22:06.998899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":113280,"kb_used_data":7360,"kb_used_omap":0,"kb_used_meta":105856,"kb_avail":754861440,"statfs":{"total":773094113280,"available":772978114560,"internally_reserved":0,"allocated":7536640,"data_stored":3991314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":108396544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-78,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001304"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.781077+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:22:04.781077+0000","last_peered":"2023-11-28T11:22:04.781077+0000","last_clean":"2023-11-28T11:22:04.781077+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:22:04.781077+0000","last_undegraded":"2023-11-28T11:22:04.781077+0000","last_fullsized":"2023-11-28T11:22:04.781077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.898460+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:22:03.898460+0000","last_peered":"2023-11-28T11:22:03.898460+0000","last_clean":"2023-11-28T11:22:03.898460+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:22:03.898460+0000","last_undegraded":"2023-11-28T11:22:03.898460+0000","last_fullsized":"2023-11-28T11:22:03.898460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:22:01.649406+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:22:01.649406+0000","last_peered":"2023-11-28T11:22:01.649406+0000","last_clean":"2023-11-28T11:22:01.649406+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:22:01.649406+0000","last_undegraded":"2023-11-28T11:22:01.649406+0000","last_fullsized":"2023-11-28T11:22:01.649406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.897644+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:22:03.897644+0000","last_peered":"2023-11-28T11:22:03.897644+0000","last_clean":"2023-11-28T11:22:03.897644+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:22:03.897644+0000","last_undegraded":"2023-11-28T11:22:03.897644+0000","last_fullsized":"2023-11-28T11:22:03.897644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424583+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:22:04.424583+0000","last_peered":"2023-11-28T11:22:04.424583+0000","last_clean":"2023-11-28T11:22:04.424583+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:22:04.424583+0000","last_undegraded":"2023-11-28T11:22:04.424583+0000","last_fullsized":"2023-11-28T11:22:04.424583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424494+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:22:04.424494+0000","last_peered":"2023-11-28T11:22:04.424494+0000","last_clean":"2023-11-28T11:22:04.424494+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:22:04.424494+0000","last_undegraded":"2023-11-28T11:22:04.424494+0000","last_fullsized":"2023-11-28T11:22:04.424494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.425950+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:22:04.425950+0000","last_peered":"2023-11-28T11:22:04.425950+0000","last_clean":"2023-11-28T11:22:04.425950+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:22:04.425950+0000","last_undegraded":"2023-11-28T11:22:04.425950+0000","last_fullsized":"2023-11-28T11:22:04.425950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.263216+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:22:05.263216+0000","last_peered":"2023-11-28T11:22:05.263216+0000","last_clean":"2023-11-28T11:22:05.263216+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:22:05.263216+0000","last_undegraded":"2023-11-28T11:22:05.263216+0000","last_fullsized":"2023-11-28T11:22:05.263216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.257573+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:22:05.257573+0000","last_peered":"2023-11-28T11:22:05.257573+0000","last_clean":"2023-11-28T11:22:05.257573+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:22:05.257573+0000","last_undegraded":"2023-11-28T11:22:05.257573+0000","last_fullsized":"2023-11-28T11:22:05.257573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15128,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356712,"statfs":{"total":96636764160,"available":96621273088,"internally_reserved":0,"allocated":1392640,"data_stored":943512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.87,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.996}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.204,"5min":2.23,"15min":2.23},"last":0.85699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.147,"15min":1.147},"last":0.78000000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.56499999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70699999999999996,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.625},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0269999999999999,"5min":1.627,"15min":1.627},"last":0.80800000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.876,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13904,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357936,"statfs":{"total":96636764160,"available":96622526464,"internally_reserved":0,"allocated":794624,"data_stored":352180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94999999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85299999999999998,"5min":1.181,"15min":1.181},"last":0.94699999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":1.774,"15min":1.774},"last":1.143}]},{"osd":6,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96699999999999997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.97399999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14932,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356908,"statfs":{"total":96636764160,"available":96621473792,"internally_reserved":0,"allocated":1388544,"data_stored":942493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":1.052,"15min":1.052},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.74399999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.503},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0249999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79100000000000004,"15min":0.80800000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2729999999999999,"5min":1.458,"15min":1.458},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.84199999999999997,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1639999999999999,"5min":1.591,"15min":1.591},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.64100000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.208,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.046,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.02}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13620,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358220,"statfs":{"total":96636764160,"available":96622817280,"internally_reserved":0,"allocated":765952,"data_stored":340553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.58099999999999996,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63200000000000001,"5min":1.04,"15min":1.04},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58299999999999996,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":4.093,"15min":4.093},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68200000000000005,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":3.956,"15min":3.956},"last":0.41699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76100000000000001,"15min":0.81899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0369999999999999,"5min":2.028,"15min":2.028},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.147,"15min":1.147},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.95199999999999996},"min":{"1min":0.58399999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.024,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0880000000000001,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.90800000000000003,"15min":1.0249999999999999},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.177,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80500000000000005,"15min":0.85499999999999998},"min":{"1min":0.65900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":1.272,"15min":1.272},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.86899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.56200000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14036,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357804,"statfs":{"total":96636764160,"available":96622391296,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48799999999999999,"15min":0.54300000000000004},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59999999999999998,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":3.004,"15min":3.004},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57399999999999995,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69599999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":3.274,"15min":3.274},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.034,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.89400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.93300000000000005},"min":{"1min":0.66700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0860000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.077,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.90700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":14100,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357740,"statfs":{"total":96636764160,"available":96622325760,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97699999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.746}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96899999999999997,"5min":1.847,"15min":1.847},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":1.615,"15min":1.615},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.099,"5min":1.127,"15min":1.127},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.083,"5min":1.089,"15min":1.089},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.127,"5min":1.296,"15min":1.296},"last":0.495},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.212,"15min":1.212},"last":0.52600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13908,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357932,"statfs":{"total":96636764160,"available":96622522368,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92900000000000005,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.72599999999999998},"min":{"1min":0.54800000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.67600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97699999999999998,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":2.661,"15min":2.661},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.73299999999999998},"min":{"1min":0.53700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86099999999999999,"5min":5.375,"15min":5.375},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.333,"15min":1.333},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:08.163 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:22:08.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.364+0000 7f43733ff700 1 Processor -- start 2023-11-28T11:22:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.364+0000 7f43733ff700 1 -- start start 2023-11-28T11:22:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 0x7f436c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f436c0d2320 0x7f436c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0b5800 con 0x7f436c0d2320 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0d1950 con 0x7f436c161340 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f43733ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0b5520 con 0x7f436c15c9b0 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f437199c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f436c0d2320 0x7f436c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f437199c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f436c0d2320 0x7f436c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58894/0 (socket says 172.21.15.53:58894) 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f437199c700 1 -- 172.21.15.53:0/2107983045 learned_addr learned my addr 172.21.15.53:0/2107983045 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.365+0000 7f437119b700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437099a700 1 --2- 172.21.15.53:0/2107983045 >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 0x7f436c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 -- 172.21.15.53:0/2107983045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 msgr2=0x7f436c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 -- 172.21.15.53:0/2107983045 >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 msgr2=0x7f436c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 --2- 172.21.15.53:0/2107983045 >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 0x7f436c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 -- 172.21.15.53:0/2107983045 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c0d1300 con 0x7f436c0d2320 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437099a700 1 --2- 172.21.15.53:0/2107983045 >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 0x7f436c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f437199c700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c0d2320 0x7f436c0cd7c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f436c06bc00 tx=0x7f436400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba98201cc86aebf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f4363fff700 1 -- 172.21.15.53:0/2107983045 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436400e070 con 0x7f436c0d2320 2023-11-28T11:22:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f4363fff700 1 -- 172.21.15.53:0/2107983045 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436400b910 con 0x7f436c0d2320 2023-11-28T11:22:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.366+0000 7f4363fff700 1 -- 172.21.15.53:0/2107983045 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436400bcc0 con 0x7f436c0d2320 2023-11-28T11:22:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 -- 172.21.15.53:0/2107983045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c0d2320 msgr2=0x7f436c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c0d2320 0x7f436c0cd7c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f436c06bc00 tx=0x7f436400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 -- 172.21.15.53:0/2107983045 shutdown_connections 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 --2- 172.21.15.53:0/2107983045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c0d2320 0x7f436c0cd7c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 --2- 172.21.15.53:0/2107983045 >> v2:172.21.15.157:3300/0 conn(0x7f436c161340 0x7f436c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 -- 172.21.15.53:0/2107983045 >> 172.21.15.53:0/2107983045 conn(0x7f436c072600 msgr2=0x7f436c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 -- 172.21.15.53:0/2107983045 shutdown_connections 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.367+0000 7f43733ff700 1 -- 172.21.15.53:0/2107983045 wait complete. 2023-11-28T11:22:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.369+0000 7f43733ff700 1 Processor -- start 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.369+0000 7f43733ff700 1 -- start start 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436c0d2320 0x7f436c159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0c9840 con 0x7f436c161340 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1f26a0 con 0x7f436c0d2320 2023-11-28T11:22:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f43733ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c155ff0 con 0x7f436c15c9b0 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f437199c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f437199c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58904/0 (socket says 172.21.15.53:58904) 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f437199c700 1 -- 172.21.15.53:0/1212925388 learned_addr learned my addr 172.21.15.53:0/1212925388 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f437119b700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436c0d2320 0x7f436c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.370+0000 7f437099a700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 -- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 msgr2=0x7f436c155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 -- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436c0d2320 msgr2=0x7f436c159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436c0d2320 0x7f436c159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 -- 172.21.15.53:0/1212925388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364009b30 con 0x7f436c161340 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f437199c700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f436c0675d0 tx=0x7f4364004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640099b0 con 0x7f436c161340 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640092e0 con 0x7f436c161340 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364004470 con 0x7f436c161340 2023-11-28T11:22:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.371+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c0b5130 con 0x7f436c161340 2023-11-28T11:22:08.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.372+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c0d15e0 con 0x7f436c161340 2023-11-28T11:22:08.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.372+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c0b52c0 con 0x7f436c161340 2023-11-28T11:22:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.373+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43640045d0 con 0x7f436c161340 2023-11-28T11:22:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.375+0000 7f4361ffb700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 0x7f435006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.375+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4364086aa0 con 0x7f436c161340 2023-11-28T11:22:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.375+0000 7f437119b700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 0x7f435006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.376+0000 7f437119b700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 0x7f435006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f436c15a180 tx=0x7f4368009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.379+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43640536b0 con 0x7f436c161340 2023-11-28T11:22:08.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.575+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f436c1f34a0 con 0x7f43500682f0 2023-11-28T11:22:08.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.579+0000 7f4361ffb700 1 -- 172.21.15.53:0/1212925388 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73015 (secure 0 0 0) 0x7f436c1f34a0 con 0x7f43500682f0 2023-11-28T11:22:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 msgr2=0x7f435006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 0x7f435006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f436c15a180 tx=0x7f4368009210 comp rx=0 tx=0).stop 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 msgr2=0x7f436c155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f436c0675d0 tx=0x7f4364004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 shutdown_connections 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f43500682f0 0x7f435006a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f436c161340 0x7f436c155950 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436c0d2320 0x7f436c159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 --2- 172.21.15.53:0/1212925388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f436c15c9b0 0x7f436c155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.581+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 >> 172.21.15.53:0/1212925388 conn(0x7f436c072600 msgr2=0x7f436c06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.582+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 shutdown_connections 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.582+0000 7f43733ff700 1 -- 172.21.15.53:0/1212925388 wait complete. 2023-11-28T11:22:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:22:08.601 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2023-11-28T11:22:06.998899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":113280,"kb_used_data":7360,"kb_used_omap":0,"kb_used_meta":105856,"kb_avail":754861440,"statfs":{"total":773094113280,"available":772978114560,"internally_reserved":0,"allocated":7536640,"data_stored":3991314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":108396544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-78,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001304"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.781077+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:22:04.781077+0000","last_peered":"2023-11-28T11:22:04.781077+0000","last_clean":"2023-11-28T11:22:04.781077+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:22:04.781077+0000","last_undegraded":"2023-11-28T11:22:04.781077+0000","last_fullsized":"2023-11-28T11:22:04.781077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.898460+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:22:03.898460+0000","last_peered":"2023-11-28T11:22:03.898460+0000","last_clean":"2023-11-28T11:22:03.898460+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:22:03.898460+0000","last_undegraded":"2023-11-28T11:22:03.898460+0000","last_fullsized":"2023-11-28T11:22:03.898460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:22:01.649406+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:22:01.649406+0000","last_peered":"2023-11-28T11:22:01.649406+0000","last_clean":"2023-11-28T11:22:01.649406+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:22:01.649406+0000","last_undegraded":"2023-11-28T11:22:01.649406+0000","last_fullsized":"2023-11-28T11:22:01.649406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.897644+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:22:03.897644+0000","last_peered":"2023-11-28T11:22:03.897644+0000","last_clean":"2023-11-28T11:22:03.897644+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:22:03.897644+0000","last_undegraded":"2023-11-28T11:22:03.897644+0000","last_fullsized":"2023-11-28T11:22:03.897644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424583+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:22:04.424583+0000","last_peered":"2023-11-28T11:22:04.424583+0000","last_clean":"2023-11-28T11:22:04.424583+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:22:04.424583+0000","last_undegraded":"2023-11-28T11:22:04.424583+0000","last_fullsized":"2023-11-28T11:22:04.424583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424494+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:22:04.424494+0000","last_peered":"2023-11-28T11:22:04.424494+0000","last_clean":"2023-11-28T11:22:04.424494+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:22:04.424494+0000","last_undegraded":"2023-11-28T11:22:04.424494+0000","last_fullsized":"2023-11-28T11:22:04.424494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.425950+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:22:04.425950+0000","last_peered":"2023-11-28T11:22:04.425950+0000","last_clean":"2023-11-28T11:22:04.425950+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:22:04.425950+0000","last_undegraded":"2023-11-28T11:22:04.425950+0000","last_fullsized":"2023-11-28T11:22:04.425950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.263216+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:22:05.263216+0000","last_peered":"2023-11-28T11:22:05.263216+0000","last_clean":"2023-11-28T11:22:05.263216+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:22:05.263216+0000","last_undegraded":"2023-11-28T11:22:05.263216+0000","last_fullsized":"2023-11-28T11:22:05.263216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.257573+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:22:05.257573+0000","last_peered":"2023-11-28T11:22:05.257573+0000","last_clean":"2023-11-28T11:22:05.257573+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:22:05.257573+0000","last_undegraded":"2023-11-28T11:22:05.257573+0000","last_fullsized":"2023-11-28T11:22:05.257573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15128,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356712,"statfs":{"total":96636764160,"available":96621273088,"internally_reserved":0,"allocated":1392640,"data_stored":943512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.87,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.996}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.204,"5min":2.23,"15min":2.23},"last":0.85699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.147,"15min":1.147},"last":0.78000000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.56499999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70699999999999996,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.625},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0269999999999999,"5min":1.627,"15min":1.627},"last":0.80800000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.876,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13904,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357936,"statfs":{"total":96636764160,"available":96622526464,"internally_reserved":0,"allocated":794624,"data_stored":352180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94999999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85299999999999998,"5min":1.181,"15min":1.181},"last":0.94699999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":1.774,"15min":1.774},"last":1.143}]},{"osd":6,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96699999999999997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.97399999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14932,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356908,"statfs":{"total":96636764160,"available":96621473792,"internally_reserved":0,"allocated":1388544,"data_stored":942493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":1.052,"15min":1.052},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.74399999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.503},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0249999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79100000000000004,"15min":0.80800000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2729999999999999,"5min":1.458,"15min":1.458},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.84199999999999997,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1639999999999999,"5min":1.591,"15min":1.591},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.64100000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.208,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.046,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.02}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13620,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358220,"statfs":{"total":96636764160,"available":96622817280,"internally_reserved":0,"allocated":765952,"data_stored":340553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.58099999999999996,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63200000000000001,"5min":1.04,"15min":1.04},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58299999999999996,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":4.093,"15min":4.093},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68200000000000005,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":3.956,"15min":3.956},"last":0.41699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76100000000000001,"15min":0.81899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0369999999999999,"5min":2.028,"15min":2.028},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.147,"15min":1.147},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.95199999999999996},"min":{"1min":0.58399999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.024,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0880000000000001,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.90800000000000003,"15min":1.0249999999999999},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.177,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80500000000000005,"15min":0.85499999999999998},"min":{"1min":0.65900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":1.272,"15min":1.272},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.86899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.56200000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14036,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357804,"statfs":{"total":96636764160,"available":96622391296,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48799999999999999,"15min":0.54300000000000004},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59999999999999998,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":3.004,"15min":3.004},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57399999999999995,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69599999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":3.274,"15min":3.274},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.034,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.89400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.93300000000000005},"min":{"1min":0.66700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0860000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.077,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.90700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":14100,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357740,"statfs":{"total":96636764160,"available":96622325760,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97699999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.746}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96899999999999997,"5min":1.847,"15min":1.847},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":1.615,"15min":1.615},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.099,"5min":1.127,"15min":1.127},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.083,"5min":1.089,"15min":1.089},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.127,"5min":1.296,"15min":1.296},"last":0.495},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.212,"15min":1.212},"last":0.52600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542200,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13908,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357932,"statfs":{"total":96636764160,"available":96622522368,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92900000000000005,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.72599999999999998},"min":{"1min":0.54800000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.67600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97699999999999998,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":2.661,"15min":2.661},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.73299999999999998},"min":{"1min":0.53700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86099999999999999,"5min":5.375,"15min":5.375},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.333,"15min":1.333},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:08.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:08.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.805+0000 7f397b7ff700 1 Processor -- start 2023-11-28T11:22:08.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.805+0000 7f397b7ff700 1 -- start start 2023-11-28T11:22:08.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f397b7ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39740d01b0 0x7f39740d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f397b7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39740b0230 con 0x7f39740b20b0 2023-11-28T11:22:08.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f397b7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39740b37d0 con 0x7f39740b16d0 2023-11-28T11:22:08.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f397b7ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397415cec0 con 0x7f39740d01b0 2023-11-28T11:22:08.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f397959b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39740d01b0 0x7f39740d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.806+0000 7f3978d9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39740b20b0 0x7f39740b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57036/0 (socket says 172.21.15.53:57036) 2023-11-28T11:22:08.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f3978d9a700 1 -- 172.21.15.53:0/1919036733 learned_addr learned my addr 172.21.15.53:0/1919036733 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f397959b700 1 -- 172.21.15.53:0/1919036733 >> v1:172.21.15.157:6789/0 conn(0x7f39740b16d0 legacy=0x7f39740b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f397959b700 1 -- 172.21.15.53:0/1919036733 >> v1:172.21.15.53:6789/0 conn(0x7f39740b20b0 legacy=0x7f39740b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f397959b700 1 -- 172.21.15.53:0/1919036733 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740b4570 con 0x7f39740d01b0 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f396bfff700 1 -- 172.21.15.53:0/1919036733 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393556187 0 0) 0x7f39740b0230 con 0x7f39740b20b0 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.807+0000 7f397959b700 1 --2- 172.21.15.53:0/1919036733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740d01b0 0x7f39740d0c20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3970009840 tx=0x7f3970009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bac99abb28f2d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f396bfff700 1 -- 172.21.15.53:0/1919036733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011830 con 0x7f39740d01b0 2023-11-28T11:22:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f396bfff700 1 -- 172.21.15.53:0/1919036733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970011990 con 0x7f39740d01b0 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f396bfff700 1 -- 172.21.15.53:0/1919036733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011d20 con 0x7f39740d01b0 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 -- 172.21.15.53:0/1919036733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740d01b0 msgr2=0x7f39740d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1919036733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740d01b0 0x7f39740d0c20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3970009840 tx=0x7f3970009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 -- 172.21.15.53:0/1919036733 shutdown_connections 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1919036733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740d01b0 0x7f39740d0c20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 -- 172.21.15.53:0/1919036733 >> 172.21.15.53:0/1919036733 conn(0x7f397409c3f0 msgr2=0x7f39740d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 -- 172.21.15.53:0/1919036733 shutdown_connections 2023-11-28T11:22:08.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.808+0000 7f397b7ff700 1 -- 172.21.15.53:0/1919036733 wait complete. 2023-11-28T11:22:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.810+0000 7f397b7ff700 1 Processor -- start 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.810+0000 7f397b7ff700 1 -- start start 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39740b16d0 0x7f397406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39740d01b0 0x7f39740ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397406be00 con 0x7f39740b20b0 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39740cb010 con 0x7f39740d01b0 2023-11-28T11:22:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f397b7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740cd480 con 0x7f39740b16d0 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.811+0000 7f3978d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58932/0 (socket says 172.21.15.53:58932) 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 -- 172.21.15.53:0/1451168499 learned_addr learned my addr 172.21.15.53:0/1451168499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f397959b700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39740b16d0 0x7f397406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3979d9c700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39740d01b0 0x7f39740ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 -- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39740b16d0 msgr2=0x7f397406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39740b16d0 0x7f397406b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 -- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39740d01b0 msgr2=0x7f39740ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39740d01b0 0x7f39740ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 -- 172.21.15.53:0/1451168499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700094a0 con 0x7f39740b20b0 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3978d9a700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f396400c520 tx=0x7f396400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964004c80 con 0x7f39740b20b0 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3964004e00 con 0x7f39740b20b0 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.812+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39740afbe0 con 0x7f39740b20b0 2023-11-28T11:22:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.813+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964007310 con 0x7f39740b20b0 2023-11-28T11:22:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.813+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39740b4850 con 0x7f39740b20b0 2023-11-28T11:22:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.814+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39740afec0 con 0x7f39740b20b0 2023-11-28T11:22:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.814+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f396401a020 con 0x7f39740b20b0 2023-11-28T11:22:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.816+0000 7f3969ffb700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 0x7f395806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.816+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3964085480 con 0x7f39740b20b0 2023-11-28T11:22:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.816+0000 7f397959b700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 0x7f395806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.817+0000 7f397959b700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 0x7f395806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f397001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:08.820+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3964051fe0 con 0x7f39740b20b0 2023-11-28T11:22:09.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.025+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f397415b960 con 0x7f39740b20b0 2023-11-28T11:22:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.026+0000 7f3969ffb700 1 -- 172.21.15.53:0/1451168499 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+14397 (secure 0 0 0) 0x7f397415b960 con 0x7f39740b20b0 2023-11-28T11:22:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:09.028 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":66,"fsid":"38901cf1-eb03-4cf6-8c7d-44ad237eb202","created":"2023-11-28T11:17:27.804293+0000","modified":"2023-11-28T11:22:04.894158+0000","last_up_change":"2023-11-28T11:17:40.436721+0000","last_in_change":"2023-11-28T11:17:32.362212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:17:40.990526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:17:41.611483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03c80b06-e2ed-4b09-ae77-76cbf0f42bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":132689},{"type":"v1","addr":"172.21.15.53:6814","nonce":132689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":132689},{"type":"v1","addr":"172.21.15.53:6817","nonce":132689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":132689},{"type":"v1","addr":"172.21.15.53:6824","nonce":132689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6819","nonce":132689},{"type":"v1","addr":"172.21.15.53:6821","nonce":132689}]},"public_addr":"172.21.15.53:6814/132689","cluster_addr":"172.21.15.53:6817/132689","heartbeat_back_addr":"172.21.15.53:6824/132689","heartbeat_front_addr":"172.21.15.53:6821/132689","state":["exists","up"]},{"osd":1,"uuid":"67e5f4d1-25d5-4e62-a73f-85361c4c2a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":132688},{"type":"v1","addr":"172.21.15.53:6801","nonce":132688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":132688},{"type":"v1","addr":"172.21.15.53:6803","nonce":132688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":132688},{"type":"v1","addr":"172.21.15.53:6808","nonce":132688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":132688},{"type":"v1","addr":"172.21.15.53:6805","nonce":132688}]},"public_addr":"172.21.15.53:6801/132688","cluster_addr":"172.21.15.53:6803/132688","heartbeat_back_addr":"172.21.15.53:6808/132688","heartbeat_front_addr":"172.21.15.53:6805/132688","state":["exists","up"]},{"osd":2,"uuid":"c7f8e310-8aef-4cd5-9456-a50608f17a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":132691},{"type":"v1","addr":"172.21.15.53:6809","nonce":132691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":132691},{"type":"v1","addr":"172.21.15.53:6811","nonce":132691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":132691},{"type":"v1","addr":"172.21.15.53:6820","nonce":132691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":132691},{"type":"v1","addr":"172.21.15.53:6815","nonce":132691}]},"public_addr":"172.21.15.53:6809/132691","cluster_addr":"172.21.15.53:6811/132691","heartbeat_back_addr":"172.21.15.53:6820/132691","heartbeat_front_addr":"172.21.15.53:6815/132691","state":["exists","up"]},{"osd":3,"uuid":"d0389301-550a-4c46-9f78-a5ce6c079987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6823","nonce":132695},{"type":"v1","addr":"172.21.15.53:6825","nonce":132695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":132695},{"type":"v1","addr":"172.21.15.53:6827","nonce":132695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":132695},{"type":"v1","addr":"172.21.15.53:6831","nonce":132695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":132695},{"type":"v1","addr":"172.21.15.53:6829","nonce":132695}]},"public_addr":"172.21.15.53:6825/132695","cluster_addr":"172.21.15.53:6827/132695","heartbeat_back_addr":"172.21.15.53:6831/132695","heartbeat_front_addr":"172.21.15.53:6829/132695","state":["exists","up"]},{"osd":4,"uuid":"7d8fd8c6-573a-4c8f-b45a-9bd517392dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":132917},{"type":"v1","addr":"172.21.15.157:6801","nonce":132917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":132917},{"type":"v1","addr":"172.21.15.157:6803","nonce":132917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":132917},{"type":"v1","addr":"172.21.15.157:6807","nonce":132917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":132917},{"type":"v1","addr":"172.21.15.157:6805","nonce":132917}]},"public_addr":"172.21.15.157:6801/132917","cluster_addr":"172.21.15.157:6803/132917","heartbeat_back_addr":"172.21.15.157:6807/132917","heartbeat_front_addr":"172.21.15.157:6805/132917","state":["exists","up"]},{"osd":5,"uuid":"b7ea5e07-6301-45af-899d-a751ba02d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":132918},{"type":"v1","addr":"172.21.15.157:6809","nonce":132918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":132918},{"type":"v1","addr":"172.21.15.157:6811","nonce":132918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":132918},{"type":"v1","addr":"172.21.15.157:6816","nonce":132918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":132918},{"type":"v1","addr":"172.21.15.157:6813","nonce":132918}]},"public_addr":"172.21.15.157:6809/132918","cluster_addr":"172.21.15.157:6811/132918","heartbeat_back_addr":"172.21.15.157:6816/132918","heartbeat_front_addr":"172.21.15.157:6813/132918","state":["exists","up"]},{"osd":6,"uuid":"5a77ab3e-acb3-42e9-a245-99faff511a85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":132919},{"type":"v1","addr":"172.21.15.157:6817","nonce":132919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":132919},{"type":"v1","addr":"172.21.15.157:6819","nonce":132919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":132919},{"type":"v1","addr":"172.21.15.157:6823","nonce":132919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":132919},{"type":"v1","addr":"172.21.15.157:6821","nonce":132919}]},"public_addr":"172.21.15.157:6817/132919","cluster_addr":"172.21.15.157:6819/132919","heartbeat_back_addr":"172.21.15.157:6823/132919","heartbeat_front_addr":"172.21.15.157:6821/132919","state":["exists","up"]},{"osd":7,"uuid":"4e5d1ceb-346f-40a5-8510-dc9611249e8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":132962},{"type":"v1","addr":"172.21.15.157:6825","nonce":132962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":132962},{"type":"v1","addr":"172.21.15.157:6827","nonce":132962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":132962},{"type":"v1","addr":"172.21.15.157:6831","nonce":132962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":132962},{"type":"v1","addr":"172.21.15.157:6829","nonce":132962}]},"public_addr":"172.21.15.157:6825/132962","cluster_addr":"172.21.15.157:6827/132962","heartbeat_back_addr":"172.21.15.157:6831/132962","heartbeat_front_addr":"172.21.15.157:6829/132962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.733680+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.628150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.729485+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.731975+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.919980+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.984653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.723629+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:17:37.777049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/1970232100":"2023-11-29T11:22:01.250791+0000","172.21.15.53:6833/1970232100":"2023-11-29T11:22:01.250791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 msgr2=0x7f395806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 0x7f395806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f397001c040 comp rx=0 tx=0).stop 2023-11-28T11:22:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 msgr2=0x7f397415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f396400c520 tx=0x7f396400c830 comp rx=0 tx=0).stop 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 shutdown_connections 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f39580682f0 0x7f395806a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39740b16d0 0x7f397406b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39740d01b0 0x7f39740ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 --2- 172.21.15.53:0/1451168499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39740b20b0 0x7f397415b120 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 >> 172.21.15.53:0/1451168499 conn(0x7f397409c3f0 msgr2=0x7f39741e41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 shutdown_connections 2023-11-28T11:22:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:09.029+0000 7f397b7ff700 1 -- 172.21.15.53:0/1451168499 wait complete. 2023-11-28T11:22:10.050 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:22:10.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:10.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.265+0000 7f1161fff700 1 Processor -- start 2023-11-28T11:22:10.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.265+0000 7f1161fff700 1 -- start start 2023-11-28T11:22:10.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.266+0000 7f1161fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f115c15a760 0x7f115c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.266+0000 7f1161fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 0x7f115c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.266+0000 7f1161fff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115c15e0a0 con 0x7f115c15dc30 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.266+0000 7f1161fff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c15ab30 con 0x7f115c15a760 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.266+0000 7f1161fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c160d70 con 0x7f115c0cded0 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f115c15a760 0x7f115c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f115c15a760 0x7f115c161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58934/0 (socket says 172.21.15.53:58934) 2023-11-28T11:22:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 -- 172.21.15.53:0/3422384091 learned_addr learned my addr 172.21.15.53:0/3422384091 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115bfff700 1 --2- 172.21.15.53:0/3422384091 >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 0x7f115c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 -- 172.21.15.53:0/3422384091 >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 msgr2=0x7f115c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 --2- 172.21.15.53:0/3422384091 >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 0x7f115c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 -- 172.21.15.53:0/3422384091 >> v1:172.21.15.157:6789/0 conn(0x7f115c15dc30 legacy=0x7f115c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 -- 172.21.15.53:0/3422384091 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0b5130 con 0x7f115c15a760 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115bfff700 1 --2- 172.21.15.53:0/3422384091 >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 0x7f115c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:10.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.267+0000 7f115b7fe700 1 --2- 172.21.15.53:0/3422384091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c161200 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f115c067500 tx=0x7f114c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682be0271064735f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f115a7fc700 1 -- 172.21.15.53:0/3422384091 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c004230 con 0x7f115c15a760 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f115a7fc700 1 -- 172.21.15.53:0/3422384091 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c004390 con 0x7f115c15a760 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f115a7fc700 1 -- 172.21.15.53:0/3422384091 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0036a0 con 0x7f115c15a760 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 -- 172.21.15.53:0/3422384091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 msgr2=0x7f115c161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 --2- 172.21.15.53:0/3422384091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c161200 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f115c067500 tx=0x7f114c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 -- 172.21.15.53:0/3422384091 shutdown_connections 2023-11-28T11:22:10.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 --2- 172.21.15.53:0/3422384091 >> v2:172.21.15.157:3300/0 conn(0x7f115c0cded0 0x7f115c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 --2- 172.21.15.53:0/3422384091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c161200 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 -- 172.21.15.53:0/3422384091 >> 172.21.15.53:0/3422384091 conn(0x7f115c072620 msgr2=0x7f115c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 -- 172.21.15.53:0/3422384091 shutdown_connections 2023-11-28T11:22:10.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.268+0000 7f1161fff700 1 -- 172.21.15.53:0/3422384091 wait complete. 2023-11-28T11:22:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.270+0000 7f1161fff700 1 Processor -- start 2023-11-28T11:22:10.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.270+0000 7f1161fff700 1 -- start start 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f115c0cded0 0x7f115c153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c15dc30 0x7f115c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0c79a0 con 0x7f115c15a760 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c153d50 con 0x7f115c15dc30 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f1161fff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0c6d20 con 0x7f115c0cded0 2023-11-28T11:22:10.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f115affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f115affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c15bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58938/0 (socket says 172.21.15.53:58938) 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f115affd700 1 -- 172.21.15.53:0/1990988441 learned_addr learned my addr 172.21.15.53:0/1990988441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f115b7fe700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f115c0cded0 0x7f115c153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.271+0000 7f115bfff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c15dc30 0x7f115c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 -- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f115c0cded0 msgr2=0x7f115c153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f115c0cded0 0x7f115c153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 -- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c15dc30 msgr2=0x7f115c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f115c15dc30 0x7f115c15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c009550 con 0x7f115c15a760 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f115affd700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c15bfa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f115000b4a0 tx=0x7f115000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150011830 con 0x7f115c15a760 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150011990 con 0x7f115c15a760 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150011d40 con 0x7f115c15a760 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c0b5670 con 0x7f115c15a760 2023-11-28T11:22:10.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.272+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c0b5390 con 0x7f115c15a760 2023-11-28T11:22:10.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.273+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f115c160720 con 0x7f115c15a760 2023-11-28T11:22:10.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.274+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f115000f050 con 0x7f115c15a760 2023-11-28T11:22:10.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.275+0000 7f1143fff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1148068320 0x7f114806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.275+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1150085570 con 0x7f115c15a760 2023-11-28T11:22:10.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.275+0000 7f1143fff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f114806e790 0x7f1148070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.275+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1148071270 con 0x7f114806e790 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.276+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150087b50 con 0x7f115c15a760 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.276+0000 7f115b7fe700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1148068320 0x7f114806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.276+0000 7f115bfff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f114806e790 0x7f1148070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.276+0000 7f115bfff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f114806e790 0x7f1148070ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.276+0000 7f115b7fe700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1148068320 0x7f114806a7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f114c009130 tx=0x7f114c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.277+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== osd.0 v2:172.21.15.53:6812/132689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1148071270 con 0x7f114806e790 2023-11-28T11:22:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.292+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 --> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f115c04e620 con 0x7f114806e790 2023-11-28T11:22:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.292+0000 7f1143fff700 1 -- 172.21.15.53:0/1990988441 <== osd.0 v2:172.21.15.53:6812/132689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f115c04e620 con 0x7f114806e790 2023-11-28T11:22:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f114806e790 msgr2=0x7f1148070ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:6812/132689,v1:172.21.15.53:6814/132689] conn(0x7f114806e790 0x7f1148070ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1148068320 msgr2=0x7f114806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f1148068320 0x7f114806a7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f114c009130 tx=0x7f114c008030 comp rx=0 tx=0).stop 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 msgr2=0x7f115c15bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 --2- 172.21.15.53:0/1990988441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115c15a760 0x7f115c15bfa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f115000b4a0 tx=0x7f115000b860 comp rx=0 tx=0).stop 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f115bfff700 1 -- 172.21.15.53:0/1990988441 reap_dead start 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 shutdown_connections 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 >> 172.21.15.53:0/1990988441 conn(0x7f115c072620 msgr2=0x7f115c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 shutdown_connections 2023-11-28T11:22:10.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.293+0000 7f1161fff700 1 -- 172.21.15.53:0/1990988441 wait complete. 2023-11-28T11:22:10.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:22:10.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.513+0000 7f694e1ff700 1 Processor -- start 2023-11-28T11:22:10.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.513+0000 7f694e1ff700 1 -- start start 2023-11-28T11:22:10.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f694815c9b0 0x7f69480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6948161340 0x7f69480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69480d2320 0x7f69480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480b5800 con 0x7f69480d2320 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480d1950 con 0x7f694815c9b0 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.514+0000 7f694e1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480b5520 con 0x7f6948161340 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69480d2320 0x7f69480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69480d2320 0x7f69480cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58942/0 (socket says 172.21.15.53:58942) 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 -- 172.21.15.53:0/935187665 learned_addr learned my addr 172.21.15.53:0/935187665 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f69477fe700 1 --2- 172.21.15.53:0/935187665 >> v2:172.21.15.157:3300/0 conn(0x7f694815c9b0 0x7f69480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6946ffd700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6948161340 0x7f69480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 -- 172.21.15.53:0/935187665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6948161340 msgr2=0x7f69480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6948161340 0x7f69480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 -- 172.21.15.53:0/935187665 >> v2:172.21.15.157:3300/0 conn(0x7f694815c9b0 msgr2=0x7f69480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 --2- 172.21.15.53:0/935187665 >> v2:172.21.15.157:3300/0 conn(0x7f694815c9b0 0x7f69480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 -- 172.21.15.53:0/935187665 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480d1300 con 0x7f69480d2320 2023-11-28T11:22:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.515+0000 7f6947fff700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69480d2320 0x7f69480cd7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6948067460 tx=0x7f694000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e019268ba60d041 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f69467fc700 1 -- 172.21.15.53:0/935187665 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694000e070 con 0x7f69480d2320 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f69467fc700 1 -- 172.21.15.53:0/935187665 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694000b910 con 0x7f69480d2320 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f69467fc700 1 -- 172.21.15.53:0/935187665 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694000bcc0 con 0x7f69480d2320 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 -- 172.21.15.53:0/935187665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69480d2320 msgr2=0x7f69480cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69480d2320 0x7f69480cd7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6948067460 tx=0x7f694000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 -- 172.21.15.53:0/935187665 shutdown_connections 2023-11-28T11:22:10.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69480d2320 0x7f69480cd7c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 --2- 172.21.15.53:0/935187665 >> v2:172.21.15.157:3300/0 conn(0x7f694815c9b0 0x7f69480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 --2- 172.21.15.53:0/935187665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6948161340 0x7f69480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 -- 172.21.15.53:0/935187665 >> 172.21.15.53:0/935187665 conn(0x7f6948072600 msgr2=0x7f69480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 -- 172.21.15.53:0/935187665 shutdown_connections 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.516+0000 7f694e1ff700 1 -- 172.21.15.53:0/935187665 wait complete. 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.518+0000 7f694e1ff700 1 Processor -- start 2023-11-28T11:22:10.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.518+0000 7f694e1ff700 1 -- start start 2023-11-28T11:22:10.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480d2320 0x7f69481504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6948161340 0x7f69480c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480ca090 con 0x7f694815c9b0 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481f5090 con 0x7f6948161340 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.519+0000 7f694e1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69481f46f0 con 0x7f69480d2320 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58948/0 (socket says 172.21.15.53:58948) 2023-11-28T11:22:10.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 -- 172.21.15.53:0/2940103926 learned_addr learned my addr 172.21.15.53:0/2940103926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f69477fe700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480d2320 0x7f69481504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6947fff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6948161340 0x7f69480c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 -- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480d2320 msgr2=0x7f69481504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480d2320 0x7f69481504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 -- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6948161340 msgr2=0x7f69480c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6948161340 0x7f69480c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 -- 172.21.15.53:0/2940103926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009b30 con 0x7f694815c9b0 2023-11-28T11:22:10.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.520+0000 7f6946ffd700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9b0 tx=0x7f693c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.521+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c009710 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.521+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c00ee50 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.521+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00ca50 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.521+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480b5130 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.521+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69480d15e0 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.522+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480b52c0 con 0x7f694815c9b0 2023-11-28T11:22:10.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.523+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f693c019030 con 0x7f694815c9b0 2023-11-28T11:22:10.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.524+0000 7f692ffff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 0x7f693406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.524+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f693c0858b0 con 0x7f694815c9b0 2023-11-28T11:22:10.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.525+0000 7f69477fe700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 0x7f693406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.525+0000 7f69477fe700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 0x7f693406a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6948154bb0 tx=0x7f6938009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.528+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f693c0522e0 con 0x7f694815c9b0 2023-11-28T11:22:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.733+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f69481f4860 con 0x7f69340682f0 2023-11-28T11:22:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.734+0000 7f692ffff700 1 -- 172.21.15.53:0/2940103926 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f69481f4860 con 0x7f69340682f0 2023-11-28T11:22:10.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 msgr2=0x7f693406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 0x7f693406a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6948154bb0 tx=0x7f6938009210 comp rx=0 tx=0).stop 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 msgr2=0x7f69480c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9b0 tx=0x7f693c00ecc0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 shutdown_connections 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f69340682f0 0x7f693406a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480d2320 0x7f69481504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6948161340 0x7f69480c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 --2- 172.21.15.53:0/2940103926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f694815c9b0 0x7f69480c5fd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 >> 172.21.15.53:0/2940103926 conn(0x7f6948072600 msgr2=0x7f694815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.736+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 shutdown_connections 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.737+0000 7f694e1ff700 1 -- 172.21.15.53:0/2940103926 wait complete. 2023-11-28T11:22:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:22:10.756 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:22:10.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.960+0000 7f2ee49ff700 1 Processor -- start 2023-11-28T11:22:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.960+0000 7f2ee49ff700 1 -- start start 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.961+0000 7f2ee49ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ee0161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ee49ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee015e0a0 con 0x7f2ee015a760 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ee49ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee0160d70 con 0x7f2ee015dc30 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ee49ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee015ab30 con 0x7f2ee00cded0 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ede59c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2ee015a760 0x7f2ee00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52620/0 (socket says 172.21.15.53:52620) 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ede59c700 1 -- 172.21.15.53:0/84185328 learned_addr learned my addr 172.21.15.53:0/84185328 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2ed7fff700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ee0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679603815 0 0) 0x7f2ee015e0a0 con 0x7f2ee015a760 2023-11-28T11:22:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ecc0035b0 con 0x7f2ee015a760 2023-11-28T11:22:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889300486 0 0) 0x7f2ee0160d70 con 0x7f2ee015dc30 2023-11-28T11:22:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.962+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee015e0a0 con 0x7f2ee015dc30 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219059221 0 0) 0x7f2ecc0035b0 con 0x7f2ee015a760 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 msgr2=0x7f2ee0161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ee0161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 >> v1:172.21.15.157:6789/0 conn(0x7f2ee015dc30 legacy=0x7f2ee00d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee00b5390 con 0x7f2ee015a760 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1942283146 0 0) 0x7f2ed0001670 con 0x7f2ee015a760 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 >> v1:172.21.15.53:6789/0 conn(0x7f2ee015a760 legacy=0x7f2ee00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ecc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ecc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ecc004d90 0x7f2ecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee015ab30 con 0x7f2ecc003bc0 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00bba0 con 0x7f2ecc004d90 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00bd10 con 0x7f2ee00cded0 2023-11-28T11:22:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ecc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ed7fff700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ecc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eded9d700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ecc004d90 0x7f2ecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 -- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 msgr2=0x7f2ecc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee00cded0 0x7f2ecc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 -- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ecc004d90 msgr2=0x7f2ecc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ecc004d90 0x7f2ecc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2ede59c700 1 -- 172.21.15.53:0/84185328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0035b0 con 0x7f2ecc003bc0 2023-11-28T11:22:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.963+0000 7f2eded9d700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ecc004d90 0x7f2ecc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2ede59c700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ecc003fb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0001eb0 tx=0x7f2ecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53302a26317efce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000cc70 con 0x7f2ecc003bc0 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed000cdf0 con 0x7f2ecc003bc0 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2eddd9b700 1 -- 172.21.15.53:0/84185328 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0005ba0 con 0x7f2ecc003bc0 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2ee49ff700 1 -- 172.21.15.53:0/84185328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 msgr2=0x7f2ecc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2ee49ff700 1 --2- 172.21.15.53:0/84185328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ecc003fb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0001eb0 tx=0x7f2ecc005880 comp rx=0 tx=0).stop 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.964+0000 7f2eded9d700 1 -- 172.21.15.53:0/84185328 reap_dead start 2023-11-28T11:22:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.965+0000 7f2ee49ff700 1 -- 172.21.15.53:0/84185328 shutdown_connections 2023-11-28T11:22:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.965+0000 7f2ee49ff700 1 -- 172.21.15.53:0/84185328 >> 172.21.15.53:0/84185328 conn(0x7f2ee0072620 msgr2=0x7f2ee00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.965+0000 7f2ee49ff700 1 -- 172.21.15.53:0/84185328 shutdown_connections 2023-11-28T11:22:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.965+0000 7f2ee49ff700 1 -- 172.21.15.53:0/84185328 wait complete. 2023-11-28T11:22:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.966+0000 7f2ee49ff700 1 Processor -- start 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.967+0000 7f2ee49ff700 1 -- start start 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ee00cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee014c7c0 0x7f2ee00b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee01f2450 0x7f2ee006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00c7130 con 0x7f2ecc003bc0 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01f2d70 con 0x7f2ee014c7c0 2023-11-28T11:22:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ee49ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01f3050 con 0x7f2ee01f2450 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ee00cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ee00cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55960/0 (socket says 172.21.15.53:55960) 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 -- 172.21.15.53:0/2382650175 learned_addr learned my addr 172.21.15.53:0/2382650175 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2eded9d700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee01f2450 0x7f2ee006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ed7fff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee014c7c0 0x7f2ee00b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 -- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee01f2450 msgr2=0x7f2ee006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ee01f2450 0x7f2ee006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 -- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee014c7c0 msgr2=0x7f2ee00b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee014c7c0 0x7f2ee00b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.968+0000 7f2ede59c700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0160720 con 0x7f2ecc003bc0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ede59c700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ee00cbda0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c720 tx=0x7f2ed0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0005e30 con 0x7f2ecc003bc0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00070d0 con 0x7f2ecc003bc0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee00b5670 con 0x7f2ecc003bc0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0007460 con 0x7f2ecc003bc0 2023-11-28T11:22:10.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.969+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee0160a00 con 0x7f2ecc003bc0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.970+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ee004e620 con 0x7f2ecc003bc0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.971+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ed0010070 con 0x7f2ecc003bc0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed6ffd700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ec00682f0 0x7f2ec006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ed0086a60 con 0x7f2ecc003bc0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed6ffd700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7f2ec006e760 0x7f2ec0070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ec0071240 con 0x7f2ec006e760 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed008b2a0 con 0x7f2ecc003bc0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2ed7fff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ec00682f0 0x7f2ec006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.972+0000 7f2eded9d700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7f2ec006e760 0x7f2ec0070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.973+0000 7f2eded9d700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7f2ec006e760 0x7f2ec0070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.973+0000 7f2ed7fff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ec00682f0 0x7f2ec006a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ee01f00a0 tx=0x7f2ec8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.973+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== osd.1 v2:172.21.15.53:6800/132688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2ec0071240 con 0x7f2ec006e760 2023-11-28T11:22:10.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.989+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 --> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ee00d08a0 con 0x7f2ec006e760 2023-11-28T11:22:10.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.989+0000 7f2ed6ffd700 1 -- 172.21.15.53:0/2382650175 <== osd.1 v2:172.21.15.53:6800/132688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2ee00d08a0 con 0x7f2ec006e760 2023-11-28T11:22:10.990 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:10.990 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:10.991 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:10.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.989+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7f2ec006e760 msgr2=0x7f2ec0070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.989+0000 7f2ee49ff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:6800/132688,v1:172.21.15.53:6801/132688] conn(0x7f2ec006e760 0x7f2ec0070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ec00682f0 msgr2=0x7f2ec006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f2ec00682f0 0x7f2ec006a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ee01f00a0 tx=0x7f2ec8008040 comp rx=0 tx=0).stop 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 msgr2=0x7f2ee00cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 --2- 172.21.15.53:0/2382650175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc003bc0 0x7f2ee00cbda0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c720 tx=0x7f2ed0001930 comp rx=0 tx=0).stop 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2eded9d700 1 -- 172.21.15.53:0/2382650175 reap_dead start 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 shutdown_connections 2023-11-28T11:22:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 >> 172.21.15.53:0/2382650175 conn(0x7f2ee0072620 msgr2=0x7f2ee015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 shutdown_connections 2023-11-28T11:22:10.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:10.990+0000 7f2ee49ff700 1 -- 172.21.15.53:0/2382650175 wait complete. 2023-11-28T11:22:11.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:22:11.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.209+0000 7fea43bff700 1 Processor -- start 2023-11-28T11:22:11.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.209+0000 7fea43bff700 1 -- start start 2023-11-28T11:22:11.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.210+0000 7fea43bff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea3c161150 0x7fea3c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.210+0000 7fea43bff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c06ac10 con 0x7fea3c06a7b0 2023-11-28T11:22:11.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea43bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0d0840 con 0x7fea3c15cd30 2023-11-28T11:22:11.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea43bff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c1615f0 con 0x7fea3c161150 2023-11-28T11:22:11.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4219c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fea3c06a7b0 0x7fea3c070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52630/0 (socket says 172.21.15.53:52630) 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4219c700 1 -- 172.21.15.53:0/3253557183 learned_addr learned my addr 172.21.15.53:0/3253557183 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4119a700 1 --2- 172.21.15.53:0/3253557183 >> v2:172.21.15.157:3300/0 conn(0x7fea3c161150 0x7fea3c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104650591 0 0) 0x7fea3c06ac10 con 0x7fea3c06a7b0 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea280035b0 con 0x7fea3c06a7b0 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941531616 0 0) 0x7fea3c0d0840 con 0x7fea3c15cd30 2023-11-28T11:22:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea3c06ac10 con 0x7fea3c15cd30 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4119a700 1 -- 172.21.15.53:0/3253557183 >> v1:172.21.15.157:6789/0 conn(0x7fea3c15cd30 legacy=0x7fea3c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4119a700 1 -- 172.21.15.53:0/3253557183 >> v1:172.21.15.53:6789/0 conn(0x7fea3c06a7b0 legacy=0x7fea3c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.211+0000 7fea4119a700 1 -- 172.21.15.53:0/3253557183 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0d0b20 con 0x7fea3c161150 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.212+0000 7fea4119a700 1 --2- 172.21.15.53:0/3253557183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c06a270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009870 tx=0x7fea2c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f609f371b6778377 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.212+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011830 con 0x7fea3c161150 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.212+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c011990 con 0x7fea3c161150 2023-11-28T11:22:11.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.212+0000 7fea40999700 1 -- 172.21.15.53:0/3253557183 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011d20 con 0x7fea3c161150 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 -- 172.21.15.53:0/3253557183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 msgr2=0x7fea3c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 --2- 172.21.15.53:0/3253557183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c06a270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009870 tx=0x7fea2c009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 -- 172.21.15.53:0/3253557183 shutdown_connections 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 --2- 172.21.15.53:0/3253557183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c06a270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 -- 172.21.15.53:0/3253557183 >> 172.21.15.53:0/3253557183 conn(0x7fea3c06b380 msgr2=0x7fea3c06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 -- 172.21.15.53:0/3253557183 shutdown_connections 2023-11-28T11:22:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.213+0000 7fea43bff700 1 -- 172.21.15.53:0/3253557183 wait complete. 2023-11-28T11:22:11.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.215+0000 7fea43bff700 1 Processor -- start 2023-11-28T11:22:11.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.215+0000 7fea43bff700 1 -- start start 2023-11-28T11:22:11.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 0x7fea3c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c1f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c14b050 con 0x7fea3c15cd30 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0c5050 con 0x7fea3c161150 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea43bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0c5330 con 0x7fea3c06a7b0 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55972/0 (socket says 172.21.15.53:55972) 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 -- 172.21.15.53:0/3115987032 learned_addr learned my addr 172.21.15.53:0/3115987032 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4219c700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c1f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4199b700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 0x7fea3c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 -- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 msgr2=0x7fea3c0c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 0x7fea3c0c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 -- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 msgr2=0x7fea3c1f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c1f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4119a700 1 -- 172.21.15.53:0/3115987032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094a0 con 0x7fea3c15cd30 2023-11-28T11:22:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.216+0000 7fea4199b700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 0x7fea3c0c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea4119a700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fea2c014040 tx=0x7fea2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c01aa70 con 0x7fea3c15cd30 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c01abd0 con 0x7fea3c15cd30 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c160950 con 0x7fea3c15cd30 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0243f0 con 0x7fea3c15cd30 2023-11-28T11:22:11.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.217+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c160670 con 0x7fea3c15cd30 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.218+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c160c30 con 0x7fea3c15cd30 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.219+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea2c00f050 con 0x7fea3c15cd30 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.220+0000 7fea327fc700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 0x7fea2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.220+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fea2c086ce0 con 0x7fea3c15cd30 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.221+0000 7fea4199b700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 0x7fea2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.221+0000 7fea4199b700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 0x7fea2006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea380011e0 tx=0x7fea38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.225+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea2c053770 con 0x7fea3c15cd30 2023-11-28T11:22:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.426+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fea3c0745c0 con 0x7fea200682f0 2023-11-28T11:22:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.427+0000 7fea327fc700 1 -- 172.21.15.53:0/3115987032 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fea3c0745c0 con 0x7fea200682f0 2023-11-28T11:22:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 msgr2=0x7fea2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 0x7fea2006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea380011e0 tx=0x7fea38009040 comp rx=0 tx=0).stop 2023-11-28T11:22:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 msgr2=0x7fea3c14ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fea2c014040 tx=0x7fea2c004750 comp rx=0 tx=0).stop 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 shutdown_connections 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fea200682f0 0x7fea2006a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea3c06a7b0 0x7fea3c0c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c161150 0x7fea3c1f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 --2- 172.21.15.53:0/3115987032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea3c15cd30 0x7fea3c14ab10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 >> 172.21.15.53:0/3115987032 conn(0x7fea3c06b380 msgr2=0x7fea3c070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 shutdown_connections 2023-11-28T11:22:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.429+0000 7fea43bff700 1 -- 172.21.15.53:0/3115987032 wait complete. 2023-11-28T11:22:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:22:11.448 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:22:11.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:11.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.653+0000 7fc9c6dff700 1 Processor -- start 2023-11-28T11:22:11.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.653+0000 7fc9c6dff700 1 -- start start 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c6dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c015a760 0x7fc9c0161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c6dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9c015dc30 0x7fc9c00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c6dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c015e0a0 con 0x7fc9c00cd7c0 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c6dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c015ab30 con 0x7fc9c015dc30 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c6dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0160d70 con 0x7fc9c015a760 2023-11-28T11:22:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c539c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc9c00cd7c0 0x7fc9c00d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52646/0 (socket says 172.21.15.53:52646) 2023-11-28T11:22:11.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.654+0000 7fc9c539c700 1 -- 172.21.15.53:0/349332176 learned_addr learned my addr 172.21.15.53:0/349332176 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 --2- 172.21.15.53:0/349332176 >> v2:172.21.15.157:3300/0 conn(0x7fc9c015dc30 0x7fc9c00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9c4b9b700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c015a760 0x7fc9c0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bf7fe700 1 -- 172.21.15.53:0/349332176 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505086630 0 0) 0x7fc9c015e0a0 con 0x7fc9c00cd7c0 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bf7fe700 1 -- 172.21.15.53:0/349332176 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b00035b0 con 0x7fc9c00cd7c0 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 -- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c015a760 msgr2=0x7fc9c0161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c015a760 0x7fc9c0161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 -- 172.21.15.53:0/349332176 >> v1:172.21.15.53:6789/0 conn(0x7fc9c00cd7c0 legacy=0x7fc9c00d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 -- 172.21.15.53:0/349332176 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00b5130 con 0x7fc9c015dc30 2023-11-28T11:22:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.655+0000 7fc9bffff700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c00b4bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009840 tx=0x7fc9ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=615b36de51732247 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9bf7fe700 1 -- 172.21.15.53:0/349332176 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac011830 con 0x7fc9c015dc30 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9bf7fe700 1 -- 172.21.15.53:0/349332176 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac011990 con 0x7fc9c015dc30 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9bf7fe700 1 -- 172.21.15.53:0/349332176 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac011d20 con 0x7fc9c015dc30 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 -- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 msgr2=0x7fc9c00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c00b4bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009840 tx=0x7fc9ac009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 -- 172.21.15.53:0/349332176 shutdown_connections 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c015a760 0x7fc9c0161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/349332176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c00b4bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 -- 172.21.15.53:0/349332176 >> 172.21.15.53:0/349332176 conn(0x7fc9c0072620 msgr2=0x7fc9c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 -- 172.21.15.53:0/349332176 shutdown_connections 2023-11-28T11:22:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.656+0000 7fc9c6dff700 1 -- 172.21.15.53:0/349332176 wait complete. 2023-11-28T11:22:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.658+0000 7fc9c6dff700 1 Processor -- start 2023-11-28T11:22:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.658+0000 7fc9c6dff700 1 -- start start 2023-11-28T11:22:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c00cd7c0 0x7fc9c01577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 0x7fc9c00c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0157d00 con 0x7fc9c015a760 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b6ff0 con 0x7fc9c015dc30 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9c6dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b72d0 con 0x7fc9c00cd7c0 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 0x7fc9c00c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 0x7fc9c00c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55978/0 (socket says 172.21.15.53:55978) 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.659+0000 7fc9bffff700 1 -- 172.21.15.53:0/3958926551 learned_addr learned my addr 172.21.15.53:0/3958926551 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9c4b9b700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c00cd7c0 0x7fc9c01577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9c539c700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 -- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c00cd7c0 msgr2=0x7fc9c01577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c00cd7c0 0x7fc9c01577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 -- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 msgr2=0x7fc9c015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9c015dc30 0x7fc9c015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0094a0 con 0x7fc9c015a760 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9c4b9b700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9c00cd7c0 0x7fc9c01577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bffff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 0x7fc9c00c6c80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009fd0 tx=0x7fc9ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0036a0 con 0x7fc9c015a760 2023-11-28T11:22:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac01e070 con 0x7fc9c015a760 2023-11-28T11:22:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac003a60 con 0x7fc9c015a760 2023-11-28T11:22:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00b5670 con 0x7fc9c015a760 2023-11-28T11:22:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.660+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00b5390 con 0x7fc9c015a760 2023-11-28T11:22:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.661+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9c0160720 con 0x7fc9c015a760 2023-11-28T11:22:11.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.662+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9ac00f050 con 0x7fc9c015a760 2023-11-28T11:22:11.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.663+0000 7fc9bd7fa700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a40682f0 0x7fc9a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc9ac0869b0 con 0x7fc9c015a760 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9bd7fa700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7fc9a406e760 0x7fc9a4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9a4071240 con 0x7fc9a406e760 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9ac08b2a0 con 0x7fc9c015a760 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9c4b9b700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a40682f0 0x7fc9a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9c539c700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7fc9a406e760 0x7fc9a4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.664+0000 7fc9c539c700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7fc9a406e760 0x7fc9a4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.665+0000 7fc9c4b9b700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a40682f0 0x7fc9a406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009c30 tx=0x7fc9b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.665+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== osd.2 v2:172.21.15.53:6807/132691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc9a4071240 con 0x7fc9a406e760 2023-11-28T11:22:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.680+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 --> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc9c004e620 con 0x7fc9a406e760 2023-11-28T11:22:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9bd7fa700 1 -- 172.21.15.53:0/3958926551 <== osd.2 v2:172.21.15.53:6807/132691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc9c004e620 con 0x7fc9a406e760 2023-11-28T11:22:11.682 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:11.682 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:11.682 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7fc9a406e760 msgr2=0x7fc9a4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:6807/132691,v1:172.21.15.53:6809/132691] conn(0x7fc9a406e760 0x7fc9a4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a40682f0 msgr2=0x7fc9a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fc9a40682f0 0x7fc9a406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009c30 tx=0x7fc9b4008040 comp rx=0 tx=0).stop 2023-11-28T11:22:11.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 msgr2=0x7fc9c00c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 --2- 172.21.15.53:0/3958926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c015a760 0x7fc9c00c6c80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009fd0 tx=0x7fc9ac004750 comp rx=0 tx=0).stop 2023-11-28T11:22:11.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c539c700 1 -- 172.21.15.53:0/3958926551 reap_dead start 2023-11-28T11:22:11.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 shutdown_connections 2023-11-28T11:22:11.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 >> 172.21.15.53:0/3958926551 conn(0x7fc9c0072620 msgr2=0x7fc9c015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 shutdown_connections 2023-11-28T11:22:11.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.681+0000 7fc9c6dff700 1 -- 172.21.15.53:0/3958926551 wait complete. 2023-11-28T11:22:11.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:22:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.901+0000 7fab1a9ff700 1 Processor -- start 2023-11-28T11:22:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.901+0000 7fab1a9ff700 1 -- start start 2023-11-28T11:22:11.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.902+0000 7fab1a9ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab1415c9b0 0x7fab140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.902+0000 7fab1a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab140cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.902+0000 7fab1a9ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab140d1950 con 0x7fab14161340 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.902+0000 7fab1a9ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140b5800 con 0x7fab1415c9b0 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.902+0000 7fab1a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab140ce140 con 0x7fab140cd7c0 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab1415c9b0 0x7fab140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab137fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fab14161340 0x7fab140d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52648/0 (socket says 172.21.15.53:52648) 2023-11-28T11:22:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab137fe700 1 -- 172.21.15.53:0/3771142479 learned_addr learned my addr 172.21.15.53:0/3771142479 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab18f9c700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab140cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 -- 172.21.15.53:0/3771142479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 msgr2=0x7fab140cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab140cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 -- 172.21.15.53:0/3771142479 >> v1:172.21.15.53:6789/0 conn(0x7fab14161340 legacy=0x7fab140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 -- 172.21.15.53:0/3771142479 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140d24f0 con 0x7fab1415c9b0 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab12ffd700 1 -- 172.21.15.53:0/3771142479 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911481284 0 0) 0x7fab140d1950 con 0x7fab14161340 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.903+0000 7fab13fff700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab140b4bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fab04009840 tx=0x7fab04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddef34098eeaf36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab12ffd700 1 -- 172.21.15.53:0/3771142479 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011830 con 0x7fab1415c9b0 2023-11-28T11:22:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab12ffd700 1 -- 172.21.15.53:0/3771142479 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04011990 con 0x7fab1415c9b0 2023-11-28T11:22:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab12ffd700 1 -- 172.21.15.53:0/3771142479 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011d20 con 0x7fab1415c9b0 2023-11-28T11:22:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 -- 172.21.15.53:0/3771142479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 msgr2=0x7fab140b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab140b4bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fab04009840 tx=0x7fab04009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 -- 172.21.15.53:0/3771142479 shutdown_connections 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab140cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/3771142479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab140b4bf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 -- 172.21.15.53:0/3771142479 >> 172.21.15.53:0/3771142479 conn(0x7fab14072600 msgr2=0x7fab140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 -- 172.21.15.53:0/3771142479 shutdown_connections 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.904+0000 7fab1a9ff700 1 -- 172.21.15.53:0/3771142479 wait complete. 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.906+0000 7fab1a9ff700 1 Processor -- start 2023-11-28T11:22:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.906+0000 7fab1a9ff700 1 -- start start 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab14148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 0x7fab140c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141497d0 con 0x7fab1415c9b0 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141f6d00 con 0x7fab14161340 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab1a9ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab141f6fe0 con 0x7fab140cd7c0 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55992/0 (socket says 172.21.15.53:55992) 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.907+0000 7fab137fe700 1 -- 172.21.15.53:0/4006667701 learned_addr learned my addr 172.21.15.53:0/4006667701 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab18f9c700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 0x7fab140c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 -- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 msgr2=0x7fab14148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab13fff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab14148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab14148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 -- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 msgr2=0x7fab140c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 0x7fab140c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 -- 172.21.15.53:0/4006667701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040094a0 con 0x7fab1415c9b0 2023-11-28T11:22:11.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab18f9c700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 0x7fab140c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:11.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab13fff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab14148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:11.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab137fe700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fab080011e0 tx=0x7fab0800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08004c80 con 0x7fab1415c9b0 2023-11-28T11:22:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08004e00 con 0x7fab1415c9b0 2023-11-28T11:22:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08007310 con 0x7fab1415c9b0 2023-11-28T11:22:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.908+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab140d1300 con 0x7fab1415c9b0 2023-11-28T11:22:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.909+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab140b5520 con 0x7fab1415c9b0 2023-11-28T11:22:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.910+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab140d15e0 con 0x7fab1415c9b0 2023-11-28T11:22:11.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.910+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab0801a020 con 0x7fab1415c9b0 2023-11-28T11:22:11.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.912+0000 7fab10ff9700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.912+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fab080856a0 con 0x7fab1415c9b0 2023-11-28T11:22:11.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.912+0000 7fab13fff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.913+0000 7fab13fff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 0x7fab0006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab040090b0 tx=0x7fab04008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:11.916+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab080521e0 con 0x7fab1415c9b0 2023-11-28T11:22:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.118+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fab141f71a0 con 0x7fab000682f0 2023-11-28T11:22:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.119+0000 7fab10ff9700 1 -- 172.21.15.53:0/4006667701 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fab04011b00 con 0x7fab000682f0 2023-11-28T11:22:12.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 msgr2=0x7fab0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 0x7fab0006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab040090b0 tx=0x7fab04008030 comp rx=0 tx=0).stop 2023-11-28T11:22:12.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 msgr2=0x7fab141f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fab080011e0 tx=0x7fab0800cae0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 shutdown_connections 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab140cd7c0 0x7fab14148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14161340 0x7fab140c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 --2- 172.21.15.53:0/4006667701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab1415c9b0 0x7fab141f6240 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 >> 172.21.15.53:0/4006667701 conn(0x7fab14072600 msgr2=0x7fab140b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 shutdown_connections 2023-11-28T11:22:12.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.121+0000 7fab1a9ff700 1 -- 172.21.15.53:0/4006667701 wait complete. 2023-11-28T11:22:12.124 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:22:12.140 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:22:12.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.348+0000 7f64db5ff700 1 Processor -- start 2023-11-28T11:22:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.348+0000 7f64db5ff700 1 -- start start 2023-11-28T11:22:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.349+0000 7f64db5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 0x7f64d40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.349+0000 7f64db5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d4157530 con 0x7f64d40af8e0 2023-11-28T11:22:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.349+0000 7f64db5ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40c9820 con 0x7f64d4157160 2023-11-28T11:22:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.349+0000 7f64db5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c9b00 con 0x7f64d40afd60 2023-11-28T11:22:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d8b9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f64d40af8e0 0x7f64d40caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52662/0 (socket says 172.21.15.53:52662) 2023-11-28T11:22:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d8b9a700 1 -- 172.21.15.53:0/2824902470 learned_addr learned my addr 172.21.15.53:0/2824902470 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/2824902470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 0x7f64d40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834802951 0 0) 0x7f64d4157530 con 0x7f64d40af8e0 2023-11-28T11:22:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c00057c0 con 0x7f64d40af8e0 2023-11-28T11:22:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517761402 0 0) 0x7f64d40c9820 con 0x7f64d4157160 2023-11-28T11:22:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d4157530 con 0x7f64d4157160 2023-11-28T11:22:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d9b9c700 1 -- 172.21.15.53:0/2824902470 >> v1:172.21.15.157:6789/0 conn(0x7f64d4157160 legacy=0x7f64d415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d9b9c700 1 -- 172.21.15.53:0/2824902470 >> v1:172.21.15.53:6789/0 conn(0x7f64d40af8e0 legacy=0x7f64d40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.350+0000 7f64d9b9c700 1 -- 172.21.15.53:0/2824902470 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40c9de0 con 0x7f64d40afd60 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/2824902470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 0x7f64d40cb620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009dc0 tx=0x7f64cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7039bfb4fdccc4ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00e070 con 0x7f64d40afd60 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00bb20 con 0x7f64d40afd60 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64cbfff700 1 -- 172.21.15.53:0/2824902470 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc012430 con 0x7f64d40afd60 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64db5ff700 1 -- 172.21.15.53:0/2824902470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 msgr2=0x7f64d40cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.351+0000 7f64db5ff700 1 --2- 172.21.15.53:0/2824902470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 0x7f64d40cb620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009dc0 tx=0x7f64cc00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.352+0000 7f64db5ff700 1 -- 172.21.15.53:0/2824902470 shutdown_connections 2023-11-28T11:22:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.352+0000 7f64db5ff700 1 --2- 172.21.15.53:0/2824902470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40afd60 0x7f64d40cb620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.352+0000 7f64db5ff700 1 -- 172.21.15.53:0/2824902470 >> 172.21.15.53:0/2824902470 conn(0x7f64d40b0860 msgr2=0x7f64d40b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.352+0000 7f64db5ff700 1 -- 172.21.15.53:0/2824902470 shutdown_connections 2023-11-28T11:22:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.352+0000 7f64db5ff700 1 -- 172.21.15.53:0/2824902470 wait complete. 2023-11-28T11:22:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.354+0000 7f64db5ff700 1 Processor -- start 2023-11-28T11:22:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.354+0000 7f64db5ff700 1 -- start start 2023-11-28T11:22:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40af8e0 0x7f64d415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 0x7f64d415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d4157160 0x7f64d415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4146cd0 con 0x7f64d40afd60 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d415a8c0 con 0x7f64d4157160 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64db5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d415aaa0 con 0x7f64d40af8e0 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 0x7f64d415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 0x7f64d415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56002/0 (socket says 172.21.15.53:56002) 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 -- 172.21.15.53:0/58010262 learned_addr learned my addr 172.21.15.53:0/58010262 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d939b700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40af8e0 0x7f64d415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d4157160 0x7f64d415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 -- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40af8e0 msgr2=0x7f64d415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40af8e0 0x7f64d415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 -- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d4157160 msgr2=0x7f64d415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d4157160 0x7f64d415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d8b9a700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc009a70 con 0x7f64d40afd60 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d939b700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d40af8e0 0x7f64d415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.355+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d4157160 0x7f64d415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64d8b9a700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 0x7f64d415c080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c4d0 tx=0x7f64c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40074e0 con 0x7f64d40afd60 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4007640 con 0x7f64d40afd60 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d40ca3a0 con 0x7f64d40afd60 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4005a30 con 0x7f64d40afd60 2023-11-28T11:22:12.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.356+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40ca0c0 con 0x7f64d40afd60 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.357+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64d40b0180 con 0x7f64d40afd60 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.358+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f64c4019030 con 0x7f64d40afd60 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64c9ffb700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64b80682f0 0x7f64b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f64c4084d10 con 0x7f64d40afd60 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64c9ffb700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7f64b806e760 0x7f64b8070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64b8071240 con 0x7f64b806e760 2023-11-28T11:22:12.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c4087290 con 0x7f64d40afd60 2023-11-28T11:22:12.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64d939b700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64b80682f0 0x7f64b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.359+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7f64b806e760 0x7f64b8070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.360+0000 7f64d9b9c700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7f64b806e760 0x7f64b8070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.360+0000 7f64d939b700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64b80682f0 0x7f64b806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64d00011e0 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.361+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== osd.3 v2:172.21.15.53:6823/132695 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f64b8071240 con 0x7f64b806e760 2023-11-28T11:22:12.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 --> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64d404e620 con 0x7f64b806e760 2023-11-28T11:22:12.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64c9ffb700 1 -- 172.21.15.53:0/58010262 <== osd.3 v2:172.21.15.53:6823/132695 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f64d404e620 con 0x7f64b806e760 2023-11-28T11:22:12.377 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:12.377 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:12.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7f64b806e760 msgr2=0x7f64b8070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:6823/132695,v1:172.21.15.53:6825/132695] conn(0x7f64b806e760 0x7f64b8070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64b80682f0 msgr2=0x7f64b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f64b80682f0 0x7f64b806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64d00011e0 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 msgr2=0x7f64d415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.376+0000 7f64db5ff700 1 --2- 172.21.15.53:0/58010262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d40afd60 0x7f64d415c080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c4d0 tx=0x7f64c400c890 comp rx=0 tx=0).stop 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.377+0000 7f64d9b9c700 1 -- 172.21.15.53:0/58010262 reap_dead start 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.377+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 shutdown_connections 2023-11-28T11:22:12.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.377+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 >> 172.21.15.53:0/58010262 conn(0x7f64d40b0860 msgr2=0x7f64d4158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.377+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 shutdown_connections 2023-11-28T11:22:12.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.377+0000 7f64db5ff700 1 -- 172.21.15.53:0/58010262 wait complete. 2023-11-28T11:22:12.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:22:12.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.596+0000 7f02051ff700 1 Processor -- start 2023-11-28T11:22:12.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.597+0000 7f02051ff700 1 -- start start 2023-11-28T11:22:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.598+0000 7f02051ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02000cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.598+0000 7f02051ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000d1950 con 0x7f0200161340 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f02051ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000ce140 con 0x7f020015c9b0 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f02051ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000b5800 con 0x7f02000cd7c0 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01f7fff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0200161340 0x7f02000d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52666/0 (socket says 172.21.15.53:52666) 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01f7fff700 1 -- 172.21.15.53:0/2919036782 learned_addr learned my addr 172.21.15.53:0/2919036782 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01ff59e700 1 --2- 172.21.15.53:0/2919036782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02000cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471454502 0 0) 0x7f02000d1950 con 0x7f0200161340 2023-11-28T11:22:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec0035b0 con 0x7f0200161340 2023-11-28T11:22:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01ff59e700 1 -- 172.21.15.53:0/2919036782 >> v1:172.21.15.157:6789/0 conn(0x7f020015c9b0 legacy=0x7f02000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01ff59e700 1 -- 172.21.15.53:0/2919036782 >> v1:172.21.15.53:6789/0 conn(0x7f0200161340 legacy=0x7f02000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166356438 0 0) 0x7f02000ce140 con 0x7f020015c9b0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.599+0000 7f01ff59e700 1 -- 172.21.15.53:0/2919036782 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000d24f0 con 0x7f02000cd7c0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f01ff59e700 1 --2- 172.21.15.53:0/2919036782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02000cdc00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009dc0 tx=0x7f01f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a18624e055bb78f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800e070 con 0x7f02000cd7c0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f800bb20 con 0x7f02000cd7c0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f01fe59c700 1 -- 172.21.15.53:0/2919036782 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800bed0 con 0x7f02000cd7c0 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f02051ff700 1 -- 172.21.15.53:0/2919036782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 msgr2=0x7f02000cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.600+0000 7f02051ff700 1 --2- 172.21.15.53:0/2919036782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02000cdc00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009dc0 tx=0x7f01f800b330 comp rx=0 tx=0).stop 2023-11-28T11:22:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.601+0000 7f02051ff700 1 -- 172.21.15.53:0/2919036782 shutdown_connections 2023-11-28T11:22:12.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.601+0000 7f02051ff700 1 --2- 172.21.15.53:0/2919036782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02000cdc00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.601+0000 7f02051ff700 1 -- 172.21.15.53:0/2919036782 >> 172.21.15.53:0/2919036782 conn(0x7f0200072600 msgr2=0x7f02000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.601+0000 7f02051ff700 1 -- 172.21.15.53:0/2919036782 shutdown_connections 2023-11-28T11:22:12.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.601+0000 7f02051ff700 1 -- 172.21.15.53:0/2919036782 wait complete. 2023-11-28T11:22:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.602+0000 7f02051ff700 1 Processor -- start 2023-11-28T11:22:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.603+0000 7f02051ff700 1 -- start start 2023-11-28T11:22:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.603+0000 7f02051ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02001f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.603+0000 7f02051ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f02051ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 0x7f020015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f02051ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02001f2f40 con 0x7f020015c9b0 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f02051ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02001f3220 con 0x7f0200161340 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f02051ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c7d90 con 0x7f02000cd7c0 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56010/0 (socket says 172.21.15.53:56010) 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 -- 172.21.15.53:0/3925993389 learned_addr learned my addr 172.21.15.53:0/3925993389 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01fed9d700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02001f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01ff59e700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 0x7f020015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 -- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 msgr2=0x7f02001f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02001f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 -- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 msgr2=0x7f020015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 0x7f020015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 -- 172.21.15.53:0/3925993389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8009a70 con 0x7f020015c9b0 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01ff59e700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 0x7f020015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.604+0000 7f01f7fff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c4d0 tx=0x7f01e800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.605+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8005d70 con 0x7f020015c9b0 2023-11-28T11:22:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.605+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e80070d0 con 0x7f020015c9b0 2023-11-28T11:22:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.605+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000d1300 con 0x7f020015c9b0 2023-11-28T11:22:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.605+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8007440 con 0x7f020015c9b0 2023-11-28T11:22:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.605+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000b5520 con 0x7f020015c9b0 2023-11-28T11:22:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.606+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02000d15e0 con 0x7f020015c9b0 2023-11-28T11:22:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.607+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01e8019030 con 0x7f020015c9b0 2023-11-28T11:22:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.608+0000 7f01f77fe700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 0x7f01e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.608+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f01e8084c70 con 0x7f020015c9b0 2023-11-28T11:22:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.608+0000 7f01fed9d700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 0x7f01e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.609+0000 7f01fed9d700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 0x7f01e006a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01f0000f60 tx=0x7f01f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.611+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f01e804b5e0 con 0x7f020015c9b0 2023-11-28T11:22:12.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.815+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f02000745c0 con 0x7f01e00682f0 2023-11-28T11:22:12.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.815+0000 7f01f77fe700 1 -- 172.21.15.53:0/3925993389 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f02000745c0 con 0x7f01e00682f0 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 msgr2=0x7f01e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 0x7f01e006a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01f0000f60 tx=0x7f01f0009040 comp rx=0 tx=0).stop 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 msgr2=0x7f020015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c4d0 tx=0x7f01e800c7e0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 shutdown_connections 2023-11-28T11:22:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f01e00682f0 0x7f01e006a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02000cd7c0 0x7f02001f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200161340 0x7f020015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 --2- 172.21.15.53:0/3925993389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020015c9b0 0x7f020015d190 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 >> 172.21.15.53:0/3925993389 conn(0x7f0200072600 msgr2=0x7f02000d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 shutdown_connections 2023-11-28T11:22:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:12.818+0000 7f02051ff700 1 -- 172.21.15.53:0/3925993389 wait complete. 2023-11-28T11:22:12.821 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:22:12.837 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:22:12.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:13.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.043+0000 7f0588dff700 1 Processor -- start 2023-11-28T11:22:13.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.043+0000 7f0588dff700 1 -- start start 2023-11-28T11:22:13.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f0588dff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058415c770 0x7f058415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f0588dff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05840c92a0 0x7f058415d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f0588dff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058415d5c0 con 0x7f05840ce420 2023-11-28T11:22:13.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f0588dff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0584070810 con 0x7f058415c770 2023-11-28T11:22:13.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f0588dff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840cc8c0 con 0x7f05840c92a0 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.044+0000 7f058259c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058415c770 0x7f058415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058415c770 0x7f058415cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56020/0 (socket says 172.21.15.53:56020) 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f0582d9d700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05840ce420 0x7f05840c3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52668/0 (socket says 172.21.15.53:52668) 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 -- 172.21.15.53:0/1614032469 learned_addr learned my addr 172.21.15.53:0/1614032469 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f0581d9b700 1 --2- 172.21.15.53:0/1614032469 >> v2:172.21.15.157:3300/0 conn(0x7f05840c92a0 0x7f058415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 -- 172.21.15.53:0/1614032469 >> v2:172.21.15.157:3300/0 conn(0x7f05840c92a0 msgr2=0x7f058415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 --2- 172.21.15.53:0/1614032469 >> v2:172.21.15.157:3300/0 conn(0x7f05840c92a0 0x7f058415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 -- 172.21.15.53:0/1614032469 >> v1:172.21.15.53:6789/0 conn(0x7f05840ce420 legacy=0x7f05840c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 -- 172.21.15.53:0/1614032469 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840ccba0 con 0x7f058415c770 2023-11-28T11:22:13.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058159a700 1 -- 172.21.15.53:0/1614032469 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251066621 0 0) 0x7f058415d5c0 con 0x7f05840ce420 2023-11-28T11:22:13.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058259c700 1 --2- 172.21.15.53:0/1614032469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f058415cb40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f058406f780 tx=0x7f057400ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5751e8b627c4086c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.045+0000 7f058159a700 1 -- 172.21.15.53:0/1614032469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400aea0 con 0x7f058415c770 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f058159a700 1 -- 172.21.15.53:0/1614032469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057400cb70 con 0x7f058415c770 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f058159a700 1 -- 172.21.15.53:0/1614032469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400f470 con 0x7f058415c770 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 -- 172.21.15.53:0/1614032469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 msgr2=0x7f058415cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 --2- 172.21.15.53:0/1614032469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f058415cb40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f058406f780 tx=0x7f057400ad70 comp rx=0 tx=0).stop 2023-11-28T11:22:13.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 -- 172.21.15.53:0/1614032469 shutdown_connections 2023-11-28T11:22:13.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 --2- 172.21.15.53:0/1614032469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f058415cb40 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 --2- 172.21.15.53:0/1614032469 >> v2:172.21.15.157:3300/0 conn(0x7f05840c92a0 0x7f058415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 -- 172.21.15.53:0/1614032469 >> 172.21.15.53:0/1614032469 conn(0x7f05840b0860 msgr2=0x7f05840b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 -- 172.21.15.53:0/1614032469 shutdown_connections 2023-11-28T11:22:13.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.046+0000 7f0588dff700 1 -- 172.21.15.53:0/1614032469 wait complete. 2023-11-28T11:22:13.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.049+0000 7f0588dff700 1 Processor -- start 2023-11-28T11:22:13.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.049+0000 7f0588dff700 1 -- start start 2023-11-28T11:22:13.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05840c92a0 0x7f058415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05840ce420 0x7f05840b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f05840b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058415ac10 con 0x7f058415c770 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058415a090 con 0x7f05840c92a0 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0588dff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058415a370 con 0x7f05840ce420 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0582d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f05840b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0582d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f05840b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56026/0 (socket says 172.21.15.53:56026) 2023-11-28T11:22:13.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.050+0000 7f0582d9d700 1 -- 172.21.15.53:0/2611204306 learned_addr learned my addr 172.21.15.53:0/2611204306 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f058259c700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05840c92a0 0x7f058415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0581d9b700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05840ce420 0x7f05840b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 -- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05840ce420 msgr2=0x7f05840b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05840ce420 0x7f05840b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 -- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05840c92a0 msgr2=0x7f058415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05840c92a0 0x7f058415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057400a6f0 con 0x7f058415c770 2023-11-28T11:22:13.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0581d9b700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05840ce420 0x7f05840b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:13.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0582d9d700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f05840b3b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f057c00cca0 tx=0x7f057c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c00cea0 con 0x7f058415c770 2023-11-28T11:22:13.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c00ade0 con 0x7f058415c770 2023-11-28T11:22:13.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05840cce80 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.051+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c007310 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.052+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05840ce9b0 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.052+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05840cd010 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.053+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f057c010070 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.054+0000 7f0572ffd700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05640682f0 0x7f056406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.054+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f057c085640 con 0x7f058415c770 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.054+0000 7f0572ffd700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f056406e760 0x7f0564070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.055+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0564071240 con 0x7f056406e760 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.055+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c087c20 con 0x7f058415c770 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.055+0000 7f058259c700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05640682f0 0x7f056406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.055+0000 7f0581d9b700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f056406e760 0x7f0564070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.056+0000 7f058259c700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05640682f0 0x7f056406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f057400a9e0 tx=0x7f0574012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.056+0000 7f0581d9b700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f056406e760 0x7f0564070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.057+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== osd.4 v2:172.21.15.157:6800/132917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0564071240 con 0x7f056406e760 2023-11-28T11:22:13.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.072+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 --> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f058404e620 con 0x7f056406e760 2023-11-28T11:22:13.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.072+0000 7f0572ffd700 1 -- 172.21.15.53:0/2611204306 <== osd.4 v2:172.21.15.157:6800/132917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f058404e620 con 0x7f056406e760 2023-11-28T11:22:13.073 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:13.073 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:13.073 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:13.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.072+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f056406e760 msgr2=0x7f0564070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6800/132917,v1:172.21.15.157:6801/132917] conn(0x7f056406e760 0x7f0564070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05640682f0 msgr2=0x7f056406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05640682f0 0x7f056406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f057400a9e0 tx=0x7f0574012000 comp rx=0 tx=0).stop 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 msgr2=0x7f05840b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 --2- 172.21.15.53:0/2611204306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058415c770 0x7f05840b3b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f057c00cca0 tx=0x7f057c00a340 comp rx=0 tx=0).stop 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0582d9d700 1 -- 172.21.15.53:0/2611204306 reap_dead start 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 shutdown_connections 2023-11-28T11:22:13.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 >> 172.21.15.53:0/2611204306 conn(0x7f05840b0860 msgr2=0x7f058415cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 shutdown_connections 2023-11-28T11:22:13.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.073+0000 7f0588dff700 1 -- 172.21.15.53:0/2611204306 wait complete. 2023-11-28T11:22:13.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:22:13.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.291+0000 7f49dcbff700 1 Processor -- start 2023-11-28T11:22:13.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.291+0000 7f49dcbff700 1 -- start start 2023-11-28T11:22:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.292+0000 7f49dcbff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49d815c9b0 0x7f49d80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.292+0000 7f49dcbff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49d80cd7c0 0x7f49d80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49dcbff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80d1950 con 0x7f49d8161340 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49dcbff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80b5800 con 0x7f49d80cd7c0 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49dcbff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80ce140 con 0x7f49d815c9b0 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49cffff700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f49d8161340 0x7f49d80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52684/0 (socket says 172.21.15.53:52684) 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49cffff700 1 -- 172.21.15.53:0/4139806112 learned_addr learned my addr 172.21.15.53:0/4139806112 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/4139806112 >> v2:172.21.15.53:3300/0 conn(0x7f49d80cd7c0 0x7f49d80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d659c700 1 --2- 172.21.15.53:0/4139806112 >> v2:172.21.15.157:3300/0 conn(0x7f49d815c9b0 0x7f49d80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d5d9b700 1 -- 172.21.15.53:0/4139806112 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436027424 0 0) 0x7f49d80d1950 con 0x7f49d8161340 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d5d9b700 1 -- 172.21.15.53:0/4139806112 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c40035b0 con 0x7f49d8161340 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d6d9d700 1 -- 172.21.15.53:0/4139806112 >> v2:172.21.15.157:3300/0 conn(0x7f49d815c9b0 msgr2=0x7f49d80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/4139806112 >> v2:172.21.15.157:3300/0 conn(0x7f49d815c9b0 0x7f49d80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d6d9d700 1 -- 172.21.15.53:0/4139806112 >> v1:172.21.15.53:6789/0 conn(0x7f49d8161340 legacy=0x7f49d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.293+0000 7f49d6d9d700 1 -- 172.21.15.53:0/4139806112 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80d24f0 con 0x7f49d80cd7c0 2023-11-28T11:22:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/4139806112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d80cd7c0 0x7f49d80cdc00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f49d8067460 tx=0x7f49d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c834ca1d62b2e2e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49d5d9b700 1 -- 172.21.15.53:0/4139806112 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d000e070 con 0x7f49d80cd7c0 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49d5d9b700 1 -- 172.21.15.53:0/4139806112 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d000bb20 con 0x7f49d80cd7c0 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49d5d9b700 1 -- 172.21.15.53:0/4139806112 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0012430 con 0x7f49d80cd7c0 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49dcbff700 1 -- 172.21.15.53:0/4139806112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d80cd7c0 msgr2=0x7f49d80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.294+0000 7f49dcbff700 1 --2- 172.21.15.53:0/4139806112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d80cd7c0 0x7f49d80cdc00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f49d8067460 tx=0x7f49d000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 -- 172.21.15.53:0/4139806112 shutdown_connections 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 --2- 172.21.15.53:0/4139806112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d80cd7c0 0x7f49d80cdc00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 --2- 172.21.15.53:0/4139806112 >> v2:172.21.15.157:3300/0 conn(0x7f49d815c9b0 0x7f49d80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 -- 172.21.15.53:0/4139806112 >> 172.21.15.53:0/4139806112 conn(0x7f49d8072600 msgr2=0x7f49d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 -- 172.21.15.53:0/4139806112 shutdown_connections 2023-11-28T11:22:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.295+0000 7f49dcbff700 1 -- 172.21.15.53:0/4139806112 wait complete. 2023-11-28T11:22:13.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.297+0000 7f49dcbff700 1 Processor -- start 2023-11-28T11:22:13.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.297+0000 7f49dcbff700 1 -- start start 2023-11-28T11:22:13.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80cd7c0 0x7f49d80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d815c9b0 0x7f49d80b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80c7380 con 0x7f49d8161340 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80c7660 con 0x7f49d815c9b0 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.298+0000 7f49dcbff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80c7940 con 0x7f49d80cd7c0 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56048/0 (socket says 172.21.15.53:56048) 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 -- 172.21.15.53:0/129507270 learned_addr learned my addr 172.21.15.53:0/129507270 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d659c700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80cd7c0 0x7f49d80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49cffff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d815c9b0 0x7f49d80b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 -- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80cd7c0 msgr2=0x7f49d80c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80cd7c0 0x7f49d80c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 -- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d815c9b0 msgr2=0x7f49d80b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d815c9b0 0x7f49d80b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 -- 172.21.15.53:0/129507270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009a70 con 0x7f49d8161340 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.299+0000 7f49d6d9d700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f49d0014040 tx=0x7f49d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.300+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0012430 con 0x7f49d8161340 2023-11-28T11:22:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.300+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0020070 con 0x7f49d8161340 2023-11-28T11:22:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.300+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004280 con 0x7f49d8161340 2023-11-28T11:22:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.300+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80d1300 con 0x7f49d8161340 2023-11-28T11:22:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.300+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80b5520 con 0x7f49d8161340 2023-11-28T11:22:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.301+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d80d15e0 con 0x7f49d8161340 2023-11-28T11:22:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.302+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49d002e070 con 0x7f49d8161340 2023-11-28T11:22:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.303+0000 7f49ceffd700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 0x7f49b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.303+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f49d0022070 con 0x7f49d8161340 2023-11-28T11:22:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.303+0000 7f49d659c700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 0x7f49b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.304+0000 7f49d659c700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 0x7f49b806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009b40 tx=0x7f49c8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.307+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49d00540f0 con 0x7f49d8161340 2023-11-28T11:22:13.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.506+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f49d80745c0 con 0x7f49b80682f0 2023-11-28T11:22:13.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.506+0000 7f49ceffd700 1 -- 172.21.15.53:0/129507270 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49d80745c0 con 0x7f49b80682f0 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.508+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 msgr2=0x7f49b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.508+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 0x7f49b806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009b40 tx=0x7f49c8009210 comp rx=0 tx=0).stop 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.508+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 msgr2=0x7f49d80cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.508+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f49d0014040 tx=0x7f49d0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 shutdown_connections 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f49b80682f0 0x7f49b806a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80cd7c0 0x7f49d80c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d8161340 0x7f49d80cb240 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 --2- 172.21.15.53:0/129507270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d815c9b0 0x7f49d80b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 >> 172.21.15.53:0/129507270 conn(0x7f49d8072600 msgr2=0x7f49d8160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 shutdown_connections 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.509+0000 7f49dcbff700 1 -- 172.21.15.53:0/129507270 wait complete. 2023-11-28T11:22:13.511 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:22:13.528 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:22:13.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:13.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.732+0000 7facb3bff700 1 Processor -- start 2023-11-28T11:22:13.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.732+0000 7facb3bff700 1 -- start start 2023-11-28T11:22:13.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.733+0000 7facb3bff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facac0cded0 0x7facac161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.733+0000 7facb3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.733+0000 7facb3bff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac160d70 con 0x7facac15a760 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb3bff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac15e0d0 con 0x7facac0cded0 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac15ab30 con 0x7facac15dc30 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facac0cded0 0x7facac161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb219c700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb199b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7facac15a760 0x7facac0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:45264/0 (socket says 172.21.15.53:45264) 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facac0cded0 0x7facac161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56056/0 (socket says 172.21.15.53:56056) 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb199b700 1 -- 172.21.15.53:0/15975999 learned_addr learned my addr 172.21.15.53:0/15975999 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:13.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 -- 172.21.15.53:0/15975999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 msgr2=0x7facac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 -- 172.21.15.53:0/15975999 >> v1:172.21.15.157:6789/0 conn(0x7facac15a760 legacy=0x7facac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb119a700 1 -- 172.21.15.53:0/15975999 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0b5130 con 0x7facac0cded0 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb219c700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.734+0000 7facb0999700 1 -- 172.21.15.53:0/15975999 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367585218 0 0) 0x7facac160d70 con 0x7facac15a760 2023-11-28T11:22:13.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb119a700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0cded0 0x7facac161200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe53d61b6acfcd76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb0999700 1 -- 172.21.15.53:0/15975999 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c011830 con 0x7facac0cded0 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb0999700 1 -- 172.21.15.53:0/15975999 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c011990 con 0x7facac0cded0 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb0999700 1 -- 172.21.15.53:0/15975999 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c011d20 con 0x7facac0cded0 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb3bff700 1 -- 172.21.15.53:0/15975999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0cded0 msgr2=0x7facac161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.735+0000 7facb3bff700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0cded0 0x7facac161200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 -- 172.21.15.53:0/15975999 shutdown_connections 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 --2- 172.21.15.53:0/15975999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0cded0 0x7facac161200 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 -- 172.21.15.53:0/15975999 >> 172.21.15.53:0/15975999 conn(0x7facac072620 msgr2=0x7facac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 -- 172.21.15.53:0/15975999 shutdown_connections 2023-11-28T11:22:13.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.736+0000 7facb3bff700 1 -- 172.21.15.53:0/15975999 wait complete. 2023-11-28T11:22:13.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.737+0000 7facb3bff700 1 Processor -- start 2023-11-28T11:22:13.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.737+0000 7facb3bff700 1 -- start start 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac0cded0 0x7facac158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 0x7facac0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c83c0 con 0x7facac15a760 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0b7090 con 0x7facac0cded0 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.738+0000 7facb3bff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac0b7370 con 0x7facac15dc30 2023-11-28T11:22:13.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 0x7facac0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 0x7facac0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56058/0 (socket says 172.21.15.53:56058) 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 -- 172.21.15.53:0/2830805361 learned_addr learned my addr 172.21.15.53:0/2830805361 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb199b700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac0cded0 0x7facac158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb219c700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 -- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 msgr2=0x7facac0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac15dc30 0x7facac0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 -- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac0cded0 msgr2=0x7facac158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac0cded0 0x7facac158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4009a70 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.739+0000 7facb119a700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 0x7facac0cb5d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000c00 tx=0x7fac9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0036a0 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c01f070 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c0094a0 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003a00 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac0b5670 con 0x7facac15a760 2023-11-28T11:22:13.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.740+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facac0b5390 con 0x7facac15a760 2023-11-28T11:22:13.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.742+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac9c00f050 con 0x7facac15a760 2023-11-28T11:22:13.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.742+0000 7faca27fc700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac900682f0 0x7fac9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fac9c021070 con 0x7facac15a760 2023-11-28T11:22:13.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7faca27fc700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fac9006e760 0x7fac90070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac90071240 con 0x7fac9006e760 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac9c087890 con 0x7facac15a760 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7facb199b700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac900682f0 0x7fac9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.743+0000 7facb219c700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fac9006e760 0x7fac90070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.744+0000 7facb199b700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac900682f0 0x7fac9006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faca8000f60 tx=0x7faca8007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.744+0000 7facb219c700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fac9006e760 0x7fac90070b70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.745+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== osd.5 v2:172.21.15.157:6808/132918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac90071240 con 0x7fac9006e760 2023-11-28T11:22:13.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.761+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 --> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7facac160720 con 0x7fac9006e760 2023-11-28T11:22:13.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.761+0000 7faca27fc700 1 -- 172.21.15.53:0/2830805361 <== osd.5 v2:172.21.15.157:6808/132918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7facac160720 con 0x7fac9006e760 2023-11-28T11:22:13.763 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:13.763 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:13.763 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:13.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fac9006e760 msgr2=0x7fac90070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6808/132918,v1:172.21.15.157:6809/132918] conn(0x7fac9006e760 0x7fac90070b70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac900682f0 msgr2=0x7fac9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7fac900682f0 0x7fac9006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faca8000f60 tx=0x7faca8007c00 comp rx=0 tx=0).stop 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 msgr2=0x7facac0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 --2- 172.21.15.53:0/2830805361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac15a760 0x7facac0cb5d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000c00 tx=0x7fac9c004750 comp rx=0 tx=0).stop 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb219c700 1 -- 172.21.15.53:0/2830805361 reap_dead start 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 shutdown_connections 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 >> 172.21.15.53:0/2830805361 conn(0x7facac072620 msgr2=0x7facac0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 shutdown_connections 2023-11-28T11:22:13.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.762+0000 7facb3bff700 1 -- 172.21.15.53:0/2830805361 wait complete. 2023-11-28T11:22:13.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:22:13.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.983+0000 7faf923ff700 1 Processor -- start 2023-11-28T11:22:13.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.983+0000 7faf923ff700 1 -- start start 2023-11-28T11:22:13.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.984+0000 7faf923ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.984+0000 7faf923ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf8c161340 0x7faf8c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.984+0000 7faf923ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8c0d1950 con 0x7faf8c0d2320 2023-11-28T11:22:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.984+0000 7faf923ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8c0b5800 con 0x7faf8c161340 2023-11-28T11:22:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.984+0000 7faf923ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0b5520 con 0x7faf8c15c9b0 2023-11-28T11:22:13.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.53:36586/0 (socket says 172.21.15.53:36586) 2023-11-28T11:22:13.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8bfff700 1 -- 172.21.15.53:0/472189878 learned_addr learned my addr 172.21.15.53:0/472189878 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf9099c700 1 --1- 172.21.15.53:0/472189878 >> v1:172.21.15.157:6789/0 conn(0x7faf8c0d2320 0x7faf8c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.53:45274/0 (socket says 172.21.15.53:45274) 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/472189878 >> v2:172.21.15.157:3300/0 conn(0x7faf8c161340 0x7faf8c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8affd700 1 -- 172.21.15.53:0/472189878 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107809539 0 0) 0x7faf8c0d1950 con 0x7faf8c0d2320 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8affd700 1 -- 172.21.15.53:0/472189878 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf740035b0 con 0x7faf8c0d2320 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8b7fe700 1 -- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 msgr2=0x7faf8c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8b7fe700 1 -- 172.21.15.53:0/472189878 >> v1:172.21.15.157:6789/0 conn(0x7faf8c0d2320 legacy=0x7faf8c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8b7fe700 1 -- 172.21.15.53:0/472189878 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0d1300 con 0x7faf8c161340 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.985+0000 7faf8bfff700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.986+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0d1de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf80009840 tx=0x7faf80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81dfcd4010bb6543 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.986+0000 7faf8affd700 1 -- 172.21.15.53:0/472189878 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80011830 con 0x7faf8c161340 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.986+0000 7faf8affd700 1 -- 172.21.15.53:0/472189878 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf80011990 con 0x7faf8c161340 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.986+0000 7faf8affd700 1 -- 172.21.15.53:0/472189878 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80011d20 con 0x7faf8c161340 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.986+0000 7faf923ff700 1 -- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 msgr2=0x7faf8c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0d1de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf80009840 tx=0x7faf80009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 -- 172.21.15.53:0/472189878 shutdown_connections 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 --2- 172.21.15.53:0/472189878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0d1de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 -- 172.21.15.53:0/472189878 >> 172.21.15.53:0/472189878 conn(0x7faf8c072600 msgr2=0x7faf8c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 -- 172.21.15.53:0/472189878 shutdown_connections 2023-11-28T11:22:13.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.987+0000 7faf923ff700 1 -- 172.21.15.53:0/472189878 wait complete. 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.989+0000 7faf923ff700 1 Processor -- start 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.989+0000 7faf923ff700 1 -- start start 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c151b90 con 0x7faf8c0d2320 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c151610 con 0x7faf8c161340 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf923ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0b6140 con 0x7faf8c15c9b0 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56072/0 (socket says 172.21.15.53:56072) 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 -- 172.21.15.53:0/2941604846 learned_addr learned my addr 172.21.15.53:0/2941604846 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:13.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf9099c700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 -- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 msgr2=0x7faf8c0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 -- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 msgr2=0x7faf8c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.990+0000 7faf8bfff700 1 -- 172.21.15.53:0/2941604846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c009550 con 0x7faf8c0d2320 2023-11-28T11:22:13.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf8bfff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ebe0 tx=0x7faf7c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00c8b0 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c00ca10 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00cd80 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf800094a0 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.991+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c0b5130 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.992+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c0d15e0 con 0x7faf8c0d2320 2023-11-28T11:22:13.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.993+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf7c019030 con 0x7faf8c0d2320 2023-11-28T11:22:13.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.994+0000 7faf88ff9700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 0x7faf7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.994+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faf7c085b30 con 0x7faf8c0d2320 2023-11-28T11:22:13.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.995+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 0x7faf7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.996+0000 7faf8b7fe700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 0x7faf7806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf80008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:13.998+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf7c052560 con 0x7faf8c0d2320 2023-11-28T11:22:14.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.198+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7faf8c0b52c0 con 0x7faf780682f0 2023-11-28T11:22:14.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.199+0000 7faf88ff9700 1 -- 172.21.15.53:0/2941604846 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faf8c0b52c0 con 0x7faf780682f0 2023-11-28T11:22:14.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 msgr2=0x7faf7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 0x7faf7806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf80008030 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 msgr2=0x7faf8c0b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ebe0 tx=0x7faf7c009e70 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 shutdown_connections 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7faf780682f0 0x7faf7806a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf8c161340 0x7faf8c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c0d2320 0x7faf8c0b6620 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 --2- 172.21.15.53:0/2941604846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf8c15c9b0 0x7faf8c0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 >> 172.21.15.53:0/2941604846 conn(0x7faf8c072600 msgr2=0x7faf8c160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 shutdown_connections 2023-11-28T11:22:14.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.201+0000 7faf923ff700 1 -- 172.21.15.53:0/2941604846 wait complete. 2023-11-28T11:22:14.204 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:22:14.220 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:22:14.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:14.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.424+0000 7f54073ff700 1 Processor -- start 2023-11-28T11:22:14.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.424+0000 7f54073ff700 1 -- start start 2023-11-28T11:22:14.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f54073ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54000cded0 0x7f5400161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f54073ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f54000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f54073ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5400160d70 con 0x7f540015a760 2023-11-28T11:22:14.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f54073ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540015e0d0 con 0x7f54000cded0 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f54073ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540015ab30 con 0x7f540015dc30 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54000cded0 0x7f5400161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540519b700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f540015a760 0x7f54000cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52686/0 (socket says 172.21.15.53:52686) 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54000cded0 0x7f5400161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56078/0 (socket says 172.21.15.53:56078) 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540519b700 1 -- 172.21.15.53:0/783383103 learned_addr learned my addr 172.21.15.53:0/783383103 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540599c700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f54000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 -- 172.21.15.53:0/783383103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 msgr2=0x7f54000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:14.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f54000b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 -- 172.21.15.53:0/783383103 >> v1:172.21.15.53:6789/0 conn(0x7f540015a760 legacy=0x7f54000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f540499a700 1 -- 172.21.15.53:0/783383103 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000b5130 con 0x7f54000cded0 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.426+0000 7f53f7fff700 1 -- 172.21.15.53:0/783383103 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759102708 0 0) 0x7f5400160d70 con 0x7f540015a760 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f540499a700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f5400161200 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009870 tx=0x7f53f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63a27f54f44a384 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f53f7fff700 1 -- 172.21.15.53:0/783383103 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0011830 con 0x7f54000cded0 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f53f7fff700 1 -- 172.21.15.53:0/783383103 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0011990 con 0x7f54000cded0 2023-11-28T11:22:14.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f53f7fff700 1 -- 172.21.15.53:0/783383103 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0011d20 con 0x7f54000cded0 2023-11-28T11:22:14.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f54073ff700 1 -- 172.21.15.53:0/783383103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 msgr2=0x7f5400161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.427+0000 7f54073ff700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f5400161200 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009870 tx=0x7f53f0009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 -- 172.21.15.53:0/783383103 shutdown_connections 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f54000b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 --2- 172.21.15.53:0/783383103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f5400161200 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 -- 172.21.15.53:0/783383103 >> 172.21.15.53:0/783383103 conn(0x7f5400072620 msgr2=0x7f54000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 -- 172.21.15.53:0/783383103 shutdown_connections 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.428+0000 7f54073ff700 1 -- 172.21.15.53:0/783383103 wait complete. 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.429+0000 7f54073ff700 1 Processor -- start 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 -- start start 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f54000c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f540015a760 0x7f54000b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f540014b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000ca590 con 0x7f54000cded0 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000b8070 con 0x7f540015a760 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.430+0000 7f54073ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54000ca900 con 0x7f540015dc30 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f54000c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f54000c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56088/0 (socket says 172.21.15.53:56088) 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 -- 172.21.15.53:0/882246353 learned_addr learned my addr 172.21.15.53:0/882246353 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540499a700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f540015a760 0x7f54000b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540599c700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f540014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 -- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 msgr2=0x7f540014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f540015dc30 0x7f540014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 -- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f540015a760 msgr2=0x7f54000b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f540015a760 0x7f54000b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00094a0 con 0x7f54000cded0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540499a700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f540015a760 0x7f54000b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.431+0000 7f540519b700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f54000c7f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0011e0 tx=0x7f53fc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.432+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc004c80 con 0x7f54000cded0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.432+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc004e00 con 0x7f54000cded0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.432+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc007310 con 0x7f54000cded0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.432+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54000b5670 con 0x7f54000cded0 2023-11-28T11:22:14.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.432+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54000b5390 con 0x7f54000cded0 2023-11-28T11:22:14.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.433+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54000b7890 con 0x7f54000cded0 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.434+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f53fc01a020 con 0x7f54000cded0 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f53f5ffb700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f53e40682f0 0x7f53e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f53fc085fb0 con 0x7f54000cded0 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f53f5ffb700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f53e406e760 0x7f53e4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53e4071240 con 0x7f53e406e760 2023-11-28T11:22:14.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53fc086360 con 0x7f54000cded0 2023-11-28T11:22:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.435+0000 7f540499a700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f53e40682f0 0x7f53e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.436+0000 7f540599c700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f53e406e760 0x7f53e4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.436+0000 7f540499a700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f53e40682f0 0x7f53e406a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54000ce2c0 tx=0x7f53f0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.436+0000 7f540599c700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f53e406e760 0x7f53e4070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.437+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== osd.6 v2:172.21.15.157:6815/132919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f53e4071240 con 0x7f53e406e760 2023-11-28T11:22:14.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.452+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 --> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f54000caa40 con 0x7f53e406e760 2023-11-28T11:22:14.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.453+0000 7f53f5ffb700 1 -- 172.21.15.53:0/882246353 <== osd.6 v2:172.21.15.157:6815/132919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f53f8009ad0 con 0x7f53e406e760 2023-11-28T11:22:14.454 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:14.455 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:14.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.453+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f53e406e760 msgr2=0x7f53e4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.453+0000 7f54073ff700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6815/132919,v1:172.21.15.157:6817/132919] conn(0x7f53e406e760 0x7f53e4070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f53e40682f0 msgr2=0x7f53e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f53e40682f0 0x7f53e406a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54000ce2c0 tx=0x7f53f0008030 comp rx=0 tx=0).stop 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 msgr2=0x7f54000c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 --2- 172.21.15.53:0/882246353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000cded0 0x7f54000c7f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0011e0 tx=0x7f53fc00cae0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f540599c700 1 -- 172.21.15.53:0/882246353 reap_dead start 2023-11-28T11:22:14.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 shutdown_connections 2023-11-28T11:22:14.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 >> 172.21.15.53:0/882246353 conn(0x7f5400072620 msgr2=0x7f54000b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 shutdown_connections 2023-11-28T11:22:14.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.454+0000 7f54073ff700 1 -- 172.21.15.53:0/882246353 wait complete. 2023-11-28T11:22:14.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:22:14.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.673+0000 7f38247ff700 1 Processor -- start 2023-11-28T11:22:14.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.673+0000 7f38247ff700 1 -- start start 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f38247ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f381c15c9b0 0x7f381c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f38247ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f381c0cd7c0 0x7f381c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f38247ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c0d1950 con 0x7f381c161340 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f38247ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0b5800 con 0x7f381c0cd7c0 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f38247ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0ce140 con 0x7f381c15c9b0 2023-11-28T11:22:14.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f3821d9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f381c161340 0x7f381c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52702/0 (socket says 172.21.15.53:52702) 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f3821d9a700 1 -- 172.21.15.53:0/4080156482 learned_addr learned my addr 172.21.15.53:0/4080156482 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f3822d9c700 1 --2- 172.21.15.53:0/4080156482 >> v2:172.21.15.53:3300/0 conn(0x7f381c0cd7c0 0x7f381c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.674+0000 7f382259b700 1 --2- 172.21.15.53:0/4080156482 >> v2:172.21.15.157:3300/0 conn(0x7f381c15c9b0 0x7f381c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3822d9c700 1 -- 172.21.15.53:0/4080156482 >> v2:172.21.15.157:3300/0 conn(0x7f381c15c9b0 msgr2=0x7f381c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3822d9c700 1 --2- 172.21.15.53:0/4080156482 >> v2:172.21.15.157:3300/0 conn(0x7f381c15c9b0 0x7f381c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3822d9c700 1 -- 172.21.15.53:0/4080156482 >> v1:172.21.15.53:6789/0 conn(0x7f381c161340 legacy=0x7f381c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3822d9c700 1 -- 172.21.15.53:0/4080156482 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0d24f0 con 0x7f381c0cd7c0 2023-11-28T11:22:14.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3822d9c700 1 --2- 172.21.15.53:0/4080156482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c0cd7c0 0x7f381c0cdc00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f381c067460 tx=0x7f381400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371e21abd9f17b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3821599700 1 -- 172.21.15.53:0/4080156482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381400e070 con 0x7f381c0cd7c0 2023-11-28T11:22:14.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3821599700 1 -- 172.21.15.53:0/4080156482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381400bb20 con 0x7f381c0cd7c0 2023-11-28T11:22:14.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.675+0000 7f3821599700 1 -- 172.21.15.53:0/4080156482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814012430 con 0x7f381c0cd7c0 2023-11-28T11:22:14.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 -- 172.21.15.53:0/4080156482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c0cd7c0 msgr2=0x7f381c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 --2- 172.21.15.53:0/4080156482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c0cd7c0 0x7f381c0cdc00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f381c067460 tx=0x7f381400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 -- 172.21.15.53:0/4080156482 shutdown_connections 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 --2- 172.21.15.53:0/4080156482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c0cd7c0 0x7f381c0cdc00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 --2- 172.21.15.53:0/4080156482 >> v2:172.21.15.157:3300/0 conn(0x7f381c15c9b0 0x7f381c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 -- 172.21.15.53:0/4080156482 >> 172.21.15.53:0/4080156482 conn(0x7f381c072600 msgr2=0x7f381c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 -- 172.21.15.53:0/4080156482 shutdown_connections 2023-11-28T11:22:14.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.676+0000 7f38247ff700 1 -- 172.21.15.53:0/4080156482 wait complete. 2023-11-28T11:22:14.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.678+0000 7f38247ff700 1 Processor -- start 2023-11-28T11:22:14.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.678+0000 7f38247ff700 1 -- start start 2023-11-28T11:22:14.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 0x7f381c147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0cc9a0 con 0x7f381c161340 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0cbd90 con 0x7f381c0cd7c0 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f38247ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0cc070 con 0x7f381c15c9b0 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f382259b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56114/0 (socket says 172.21.15.53:56114) 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f382259b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.53:60276/0 (socket says 172.21.15.53:60276) 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 -- 172.21.15.53:0/3980754926 learned_addr learned my addr 172.21.15.53:0/3980754926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3821d9a700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 0x7f381c147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 -- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 msgr2=0x7f381c147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 0x7f381c147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.679+0000 7f3822d9c700 1 -- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 msgr2=0x7f381c1e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3822d9c700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3822d9c700 1 -- 172.21.15.53:0/3980754926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814009a70 con 0x7f381c161340 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3821d9a700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 0x7f381c147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f382259b700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3822d9c700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3814014040 tx=0x7f3814004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814012430 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3814020070 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814004280 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c0d1300 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.680+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c0b5520 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.681+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c0d15e0 con 0x7f381c161340 2023-11-28T11:22:14.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.682+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f381402e070 con 0x7f381c161340 2023-11-28T11:22:14.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.683+0000 7f3812ffd700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 0x7f380006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.683+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3814022070 con 0x7f381c161340 2023-11-28T11:22:14.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.684+0000 7f382259b700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 0x7f380006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.684+0000 7f382259b700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 0x7f380006a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3818009f30 tx=0x7f3818009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.687+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38140541c0 con 0x7f381c161340 2023-11-28T11:22:14.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.888+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f381c0ccb50 con 0x7f38000682f0 2023-11-28T11:22:14.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.889+0000 7f3812ffd700 1 -- 172.21.15.53:0/3980754926 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f381c0ccb50 con 0x7f38000682f0 2023-11-28T11:22:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 msgr2=0x7f380006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 0x7f380006a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3818009f30 tx=0x7f3818009210 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 msgr2=0x7f381c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3814014040 tx=0x7f3814004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 shutdown_connections 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f38000682f0 0x7f380006a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c161340 0x7f381c15e500 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f381c0cd7c0 0x7f381c1e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 --2- 172.21.15.53:0/3980754926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f381c15c9b0 0x7f381c147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 >> 172.21.15.53:0/3980754926 conn(0x7f381c072600 msgr2=0x7f381c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 shutdown_connections 2023-11-28T11:22:14.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:14.891+0000 7f38247ff700 1 -- 172.21.15.53:0/3980754926 wait complete. 2023-11-28T11:22:14.894 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:22:14.910 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:22:14.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.115+0000 7f3b473ff700 1 Processor -- start 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.115+0000 7f3b473ff700 1 -- start start 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.116+0000 7f3b473ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b4015a760 0x7f3b40161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.116+0000 7f3b473ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b400cded0 0x7f3b400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.116+0000 7f3b473ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4015e0a0 con 0x7f3b4015dc30 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.116+0000 7f3b473ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4015ab30 con 0x7f3b400cded0 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.116+0000 7f3b473ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40160d70 con 0x7f3b4015a760 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b400cded0 0x7f3b400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b400cded0 0x7f3b400b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56126/0 (socket says 172.21.15.53:56126) 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 -- 172.21.15.53:0/3046526229 learned_addr learned my addr 172.21.15.53:0/3046526229 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4519b700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b4015a760 0x7f3b40161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 -- 172.21.15.53:0/3046526229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b4015a760 msgr2=0x7f3b40161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b4015a760 0x7f3b40161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 -- 172.21.15.53:0/3046526229 >> v1:172.21.15.157:6789/0 conn(0x7f3b4015dc30 legacy=0x7f3b400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 -- 172.21.15.53:0/3046526229 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b400b5130 con 0x7f3b400cded0 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.117+0000 7f3b4599c700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b400cded0 0x7f3b400b4bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009e50 tx=0x7f3b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa046c0e88dd42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b37fff700 1 -- 172.21.15.53:0/3046526229 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800e070 con 0x7f3b400cded0 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b37fff700 1 -- 172.21.15.53:0/3046526229 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3800b910 con 0x7f3b400cded0 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b37fff700 1 -- 172.21.15.53:0/3046526229 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800bcc0 con 0x7f3b400cded0 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 -- 172.21.15.53:0/3046526229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b400cded0 msgr2=0x7f3b400b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b400cded0 0x7f3b400b4bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009e50 tx=0x7f3b3800b330 comp rx=0 tx=0).stop 2023-11-28T11:22:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 -- 172.21.15.53:0/3046526229 shutdown_connections 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b400cded0 0x7f3b400b4bf0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 --2- 172.21.15.53:0/3046526229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b4015a760 0x7f3b40161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 -- 172.21.15.53:0/3046526229 >> 172.21.15.53:0/3046526229 conn(0x7f3b40072620 msgr2=0x7f3b400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 -- 172.21.15.53:0/3046526229 shutdown_connections 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.118+0000 7f3b473ff700 1 -- 172.21.15.53:0/3046526229 wait complete. 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.120+0000 7f3b473ff700 1 Processor -- start 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.120+0000 7f3b473ff700 1 -- start start 2023-11-28T11:22:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400cded0 0x7f3b400b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 0x7f3b401533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4015dc30 0x7f3b4015b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015c700 con 0x7f3b4015a760 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b400c6e40 con 0x7f3b4015dc30 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b473ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015bbc0 con 0x7f3b400cded0 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b4499a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 0x7f3b401533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b4499a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 0x7f3b401533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56130/0 (socket says 172.21.15.53:56130) 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b4499a700 1 -- 172.21.15.53:0/2755460666 learned_addr learned my addr 172.21.15.53:0/2755460666 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b4599c700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4015dc30 0x7f3b4015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.121+0000 7f3b4519b700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400cded0 0x7f3b400b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 -- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400cded0 msgr2=0x7f3b400b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b400cded0 0x7f3b400b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 -- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4015dc30 msgr2=0x7f3b4015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4015dc30 0x7f3b4015b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b38009b30 con 0x7f3b4015a760 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b4499a700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 0x7f3b401533d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c450 tx=0x7f3b3000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300052d0 con 0x7f3b4015a760 2023-11-28T11:22:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30005430 con 0x7f3b4015a760 2023-11-28T11:22:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b400b5670 con 0x7f3b4015a760 2023-11-28T11:22:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300038c0 con 0x7f3b4015a760 2023-11-28T11:22:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.122+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b400b5390 con 0x7f3b4015a760 2023-11-28T11:22:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.123+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b40160720 con 0x7f3b4015a760 2023-11-28T11:22:15.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.124+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b30019030 con 0x7f3b4015a760 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.125+0000 7f3b35ffb700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b240682f0 0x7f3b2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.125+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3b30084c30 con 0x7f3b4015a760 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.125+0000 7f3b35ffb700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f3b2406e760 0x7f3b24070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.125+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b24071240 con 0x7f3b2406e760 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.125+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b300892a0 con 0x7f3b4015a760 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.126+0000 7f3b4519b700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b240682f0 0x7f3b2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.126+0000 7f3b4599c700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f3b2406e760 0x7f3b24070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.126+0000 7f3b4519b700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b240682f0 0x7f3b2406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009be0 tx=0x7f3b3c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.126+0000 7f3b4599c700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f3b2406e760 0x7f3b24070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.127+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== osd.7 v2:172.21.15.157:6824/132962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b24071240 con 0x7f3b2406e760 2023-11-28T11:22:15.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.142+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 --> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b4004e620 con 0x7f3b2406e760 2023-11-28T11:22:15.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b35ffb700 1 -- 172.21.15.53:0/2755460666 <== osd.7 v2:172.21.15.157:6824/132962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b4004e620 con 0x7f3b2406e760 2023-11-28T11:22:15.144 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-28T11:22:15.144 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:22:15.145 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-28T11:22:15.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f3b2406e760 msgr2=0x7f3b24070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6824/132962,v1:172.21.15.157:6825/132962] conn(0x7f3b2406e760 0x7f3b24070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b240682f0 msgr2=0x7f3b2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f3b240682f0 0x7f3b2406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009be0 tx=0x7f3b3c009210 comp rx=0 tx=0).stop 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 msgr2=0x7f3b401533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b473ff700 1 --2- 172.21.15.53:0/2755460666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b4015a760 0x7f3b401533d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c450 tx=0x7f3b3000c760 comp rx=0 tx=0).stop 2023-11-28T11:22:15.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.143+0000 7f3b4599c700 1 -- 172.21.15.53:0/2755460666 reap_dead start 2023-11-28T11:22:15.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.144+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 shutdown_connections 2023-11-28T11:22:15.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.144+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 >> 172.21.15.53:0/2755460666 conn(0x7f3b40072620 msgr2=0x7f3b4015f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.144+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 shutdown_connections 2023-11-28T11:22:15.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.144+0000 7f3b473ff700 1 -- 172.21.15.53:0/2755460666 wait complete. 2023-11-28T11:22:15.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:22:15.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.364+0000 7f785f5ff700 1 Processor -- start 2023-11-28T11:22:15.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.364+0000 7f785f5ff700 1 -- start start 2023-11-28T11:22:15.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785f5ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f785815c9b0 0x7f78580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785f5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7858161340 0x7f78580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785f5ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580d1950 con 0x7f78580d2320 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785f5ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580b5800 con 0x7f785815c9b0 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785f5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580b5520 con 0x7f7858161340 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785db9c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f78580d2320 0x7f78580cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52710/0 (socket says 172.21.15.53:52710) 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785db9c700 1 -- 172.21.15.53:0/435567161 learned_addr learned my addr 172.21.15.53:0/435567161 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785d39b700 1 --2- 172.21.15.53:0/435567161 >> v2:172.21.15.157:3300/0 conn(0x7f785815c9b0 0x7f78580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.365+0000 7f785cb9a700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7858161340 0x7f78580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f785d39b700 1 -- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7858161340 msgr2=0x7f78580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f785d39b700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7858161340 0x7f78580d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f784ffff700 1 -- 172.21.15.53:0/435567161 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911255212 0 0) 0x7f78580d1950 con 0x7f78580d2320 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f785d39b700 1 -- 172.21.15.53:0/435567161 >> v1:172.21.15.53:6789/0 conn(0x7f78580d2320 legacy=0x7f78580cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f785d39b700 1 -- 172.21.15.53:0/435567161 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580d1300 con 0x7f785815c9b0 2023-11-28T11:22:15.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.366+0000 7f785d39b700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785815c9b0 0x7f78580b4bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7854009840 tx=0x7f7854009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b37446afdfcb4424 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f784ffff700 1 -- 172.21.15.53:0/435567161 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011830 con 0x7f785815c9b0 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f784ffff700 1 -- 172.21.15.53:0/435567161 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7854011990 con 0x7f785815c9b0 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f784ffff700 1 -- 172.21.15.53:0/435567161 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011d20 con 0x7f785815c9b0 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 -- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785815c9b0 msgr2=0x7f78580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785815c9b0 0x7f78580b4bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7854009840 tx=0x7f7854009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 -- 172.21.15.53:0/435567161 shutdown_connections 2023-11-28T11:22:15.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785815c9b0 0x7f78580b4bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 --2- 172.21.15.53:0/435567161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7858161340 0x7f78580d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 -- 172.21.15.53:0/435567161 >> 172.21.15.53:0/435567161 conn(0x7f7858072600 msgr2=0x7f78580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 -- 172.21.15.53:0/435567161 shutdown_connections 2023-11-28T11:22:15.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.367+0000 7f785f5ff700 1 -- 172.21.15.53:0/435567161 wait complete. 2023-11-28T11:22:15.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.369+0000 7f785f5ff700 1 Processor -- start 2023-11-28T11:22:15.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.369+0000 7f785f5ff700 1 -- start start 2023-11-28T11:22:15.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.370+0000 7f785f5ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78580d2320 0x7f78580c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.370+0000 7f785f5ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f785815c9b0 0x7f7858157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785f5ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785f5ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c6410 con 0x7f7858161340 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785f5ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858153160 con 0x7f78580d2320 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785f5ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7858153440 con 0x7f785815c9b0 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56134/0 (socket says 172.21.15.53:56134) 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 -- 172.21.15.53:0/482180983 learned_addr learned my addr 172.21.15.53:0/482180983 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:15.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785d39b700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78580d2320 0x7f78580c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785cb9a700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f785815c9b0 0x7f7858157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 -- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f785815c9b0 msgr2=0x7f7858157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f785815c9b0 0x7f7858157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 -- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78580d2320 msgr2=0x7f78580c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78580d2320 0x7f78580c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 -- 172.21.15.53:0/482180983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540094a0 con 0x7f7858161340 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.371+0000 7f785db9c700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f785000cba0 tx=0x7f785000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.372+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785000aaa0 con 0x7f7858161340 2023-11-28T11:22:15.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.372+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785000ac20 con 0x7f7858161340 2023-11-28T11:22:15.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.372+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78580b5130 con 0x7f7858161340 2023-11-28T11:22:15.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.372+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500070d0 con 0x7f7858161340 2023-11-28T11:22:15.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.372+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78580d15e0 con 0x7f7858161340 2023-11-28T11:22:15.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.373+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78580b52c0 con 0x7f7858161340 2023-11-28T11:22:15.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.374+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7850010070 con 0x7f7858161340 2023-11-28T11:22:15.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.375+0000 7f784dffb700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 0x7f783c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.375+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f78500853d0 con 0x7f7858161340 2023-11-28T11:22:15.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.376+0000 7f785d39b700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 0x7f783c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.376+0000 7f785d39b700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 0x7f783c06a7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78540090b0 tx=0x7f7854008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.379+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7850051de0 con 0x7f7858161340 2023-11-28T11:22:15.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:15.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:15.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.592+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f78580b9150 con 0x7f783c068320 2023-11-28T11:22:15.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.592+0000 7f784dffb700 1 -- 172.21.15.53:0/482180983 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78580b9150 con 0x7f783c068320 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 msgr2=0x7f783c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 0x7f783c06a7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78540090b0 tx=0x7f7854008030 comp rx=0 tx=0).stop 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 msgr2=0x7f7858154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f785000cba0 tx=0x7f785000ceb0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 shutdown_connections 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f783c068320 0x7f783c06a7d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858161340 0x7f7858154200 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78580d2320 0x7f78580c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 --2- 172.21.15.53:0/482180983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f785815c9b0 0x7f7858157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.594+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 >> 172.21.15.53:0/482180983 conn(0x7f7858072600 msgr2=0x7f78580b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.595+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 shutdown_connections 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.595+0000 7f785f5ff700 1 -- 172.21.15.53:0/482180983 wait complete. 2023-11-28T11:22:15.597 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:22:15.614 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:15.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.812+0000 7f622b1ff700 1 Processor -- start 2023-11-28T11:22:15.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.813+0000 7f622b1ff700 1 -- start start 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622b1ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f622415c9b0 0x7f62240b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622b1ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6224161340 0x7f62240d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622b1ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62240d1950 con 0x7f62240d2320 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622b1ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62240b5800 con 0x7f622415c9b0 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622b1ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62240b5520 con 0x7f6224161340 2023-11-28T11:22:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f6228f9b700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f622415c9b0 0x7f62240b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622979c700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f62240d2320 0x7f62240cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52724/0 (socket says 172.21.15.53:52724) 2023-11-28T11:22:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f622979c700 1 -- 172.21.15.53:0/359207664 learned_addr learned my addr 172.21.15.53:0/359207664 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.814+0000 7f6223fff700 1 --2- 172.21.15.53:0/359207664 >> v2:172.21.15.157:3300/0 conn(0x7f6224161340 0x7f62240d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f6228f9b700 1 -- 172.21.15.53:0/359207664 >> v2:172.21.15.157:3300/0 conn(0x7f6224161340 msgr2=0x7f62240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f6228f9b700 1 --2- 172.21.15.53:0/359207664 >> v2:172.21.15.157:3300/0 conn(0x7f6224161340 0x7f62240d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f6228f9b700 1 -- 172.21.15.53:0/359207664 >> v1:172.21.15.53:6789/0 conn(0x7f62240d2320 legacy=0x7f62240cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f6228f9b700 1 -- 172.21.15.53:0/359207664 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62240d1300 con 0x7f622415c9b0 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f62237fe700 1 -- 172.21.15.53:0/359207664 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914786604 0 0) 0x7f62240d1950 con 0x7f62240d2320 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f6228f9b700 1 --2- 172.21.15.53:0/359207664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f622415c9b0 0x7f62240b4bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f62240685c0 tx=0x7f6218009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b0c3b3c7400dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f62237fe700 1 -- 172.21.15.53:0/359207664 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011830 con 0x7f622415c9b0 2023-11-28T11:22:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f62237fe700 1 -- 172.21.15.53:0/359207664 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218011990 con 0x7f622415c9b0 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.815+0000 7f62237fe700 1 -- 172.21.15.53:0/359207664 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011d20 con 0x7f622415c9b0 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 -- 172.21.15.53:0/359207664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f622415c9b0 msgr2=0x7f62240b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 --2- 172.21.15.53:0/359207664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f622415c9b0 0x7f62240b4bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f62240685c0 tx=0x7f6218009b20 comp rx=0 tx=0).stop 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 -- 172.21.15.53:0/359207664 shutdown_connections 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 --2- 172.21.15.53:0/359207664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f622415c9b0 0x7f62240b4bf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 --2- 172.21.15.53:0/359207664 >> v2:172.21.15.157:3300/0 conn(0x7f6224161340 0x7f62240d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 -- 172.21.15.53:0/359207664 >> 172.21.15.53:0/359207664 conn(0x7f6224072600 msgr2=0x7f62240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 -- 172.21.15.53:0/359207664 shutdown_connections 2023-11-28T11:22:15.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.816+0000 7f622b1ff700 1 -- 172.21.15.53:0/359207664 wait complete. 2023-11-28T11:22:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.817+0000 7f622b1ff700 1 Processor -- start 2023-11-28T11:22:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.818+0000 7f622b1ff700 1 -- start start 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f622415c9b0 0x7f6224159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 0x7f62241599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62241e88a0 con 0x7f62240d2320 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240b7d70 con 0x7f6224161340 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622b1ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62241f70c0 con 0x7f622415c9b0 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56138/0 (socket says 172.21.15.53:56138) 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 -- 172.21.15.53:0/387480349 learned_addr learned my addr 172.21.15.53:0/387480349 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 -- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f622415c9b0 msgr2=0x7f6224159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f622415c9b0 0x7f6224159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 -- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 msgr2=0x7f62241599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622979c700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 0x7f62241599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 0x7f62241599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f6228f9b700 1 -- 172.21.15.53:0/387480349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180094a0 con 0x7f62240d2320 2023-11-28T11:22:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.819+0000 7f622979c700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 0x7f62241599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f6228f9b700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f62180090b0 tx=0x7f6218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180038a0 con 0x7f62240d2320 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621801f070 con 0x7f62240d2320 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218003c20 con 0x7f62240d2320 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62240b5130 con 0x7f62240d2320 2023-11-28T11:22:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.820+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62240d15e0 con 0x7f62240d2320 2023-11-28T11:22:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.821+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62240b52c0 con 0x7f62240d2320 2023-11-28T11:22:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.822+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f621800f050 con 0x7f62240d2320 2023-11-28T11:22:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.823+0000 7f62217fa700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 0x7f620806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.823+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6218021070 con 0x7f62240d2320 2023-11-28T11:22:15.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.824+0000 7f6223fff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 0x7f620806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.824+0000 7f6223fff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 0x7f620806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6224153f60 tx=0x7f6210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:15.827+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6218053db0 con 0x7f62240d2320 2023-11-28T11:22:16.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.025+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62240745c0 con 0x7f62080682f0 2023-11-28T11:22:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.028+0000 7f62217fa700 1 -- 172.21.15.53:0/387480349 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72814 (secure 0 0 0) 0x7f62240745c0 con 0x7f62080682f0 2023-11-28T11:22:16.030 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.030+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 msgr2=0x7f620806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.030+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 0x7f620806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6224153f60 tx=0x7f6210008040 comp rx=0 tx=0).stop 2023-11-28T11:22:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.030+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 msgr2=0x7f6224150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.030+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f62180090b0 tx=0x7f6218004750 comp rx=0 tx=0).stop 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.030+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 shutdown_connections 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f62080682f0 0x7f620806a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6224161340 0x7f62241599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62240d2320 0x7f6224150190 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 --2- 172.21.15.53:0/387480349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f622415c9b0 0x7f6224159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 >> 172.21.15.53:0/387480349 conn(0x7f6224072600 msgr2=0x7f622415fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 shutdown_connections 2023-11-28T11:22:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:16.031+0000 7f622b1ff700 1 -- 172.21.15.53:0/387480349 wait complete. 2023-11-28T11:22:16.034 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:22:16.049 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-11-28T11:22:15.000746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":113376,"kb_used_data":7392,"kb_used_omap":0,"kb_used_meta":105920,"kb_avail":754861344,"statfs":{"total":773094113280,"available":772978016256,"internally_reserved":0,"allocated":7569408,"data_stored":4003905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":108462080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.099181"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.781077+0000","last_change":"2023-11-28T11:17:43.460394+0000","last_active":"2023-11-28T11:22:04.781077+0000","last_peered":"2023-11-28T11:22:04.781077+0000","last_clean":"2023-11-28T11:22:04.781077+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:22:04.781077+0000","last_undegraded":"2023-11-28T11:22:04.781077+0000","last_fullsized":"2023-11-28T11:22:04.781077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.898460+0000","last_change":"2023-11-28T11:17:43.457158+0000","last_active":"2023-11-28T11:22:03.898460+0000","last_peered":"2023-11-28T11:22:03.898460+0000","last_clean":"2023-11-28T11:22:03.898460+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:22:03.898460+0000","last_undegraded":"2023-11-28T11:22:03.898460+0000","last_fullsized":"2023-11-28T11:22:03.898460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:10.075662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:22:01.649406+0000","last_change":"2023-11-28T11:17:43.543891+0000","last_active":"2023-11-28T11:22:01.649406+0000","last_peered":"2023-11-28T11:22:01.649406+0000","last_clean":"2023-11-28T11:22:01.649406+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:22:01.649406+0000","last_undegraded":"2023-11-28T11:22:01.649406+0000","last_fullsized":"2023-11-28T11:22:01.649406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:41.186128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:03.897644+0000","last_change":"2023-11-28T11:17:43.457725+0000","last_active":"2023-11-28T11:22:03.897644+0000","last_peered":"2023-11-28T11:22:03.897644+0000","last_clean":"2023-11-28T11:22:03.897644+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:22:03.897644+0000","last_undegraded":"2023-11-28T11:22:03.897644+0000","last_fullsized":"2023-11-28T11:22:03.897644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:04.545234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424583+0000","last_change":"2023-11-28T11:17:42.455030+0000","last_active":"2023-11-28T11:22:04.424583+0000","last_peered":"2023-11-28T11:22:04.424583+0000","last_clean":"2023-11-28T11:22:04.424583+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:22:04.424583+0000","last_undegraded":"2023-11-28T11:22:04.424583+0000","last_fullsized":"2023-11-28T11:22:04.424583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:41.438817+0000","last_clean_scrub_stamp":"2023-11-28T11:17:41.438817+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:17:40.184115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.424494+0000","last_change":"2023-11-28T11:17:43.456949+0000","last_active":"2023-11-28T11:22:04.424494+0000","last_peered":"2023-11-28T11:22:04.424494+0000","last_clean":"2023-11-28T11:22:04.424494+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:22:04.424494+0000","last_undegraded":"2023-11-28T11:22:04.424494+0000","last_fullsized":"2023-11-28T11:22:04.424494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:29.243077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:22:04.425950+0000","last_change":"2023-11-28T11:17:43.460194+0000","last_active":"2023-11-28T11:22:04.425950+0000","last_peered":"2023-11-28T11:22:04.425950+0000","last_clean":"2023-11-28T11:22:04.425950+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:22:04.425950+0000","last_undegraded":"2023-11-28T11:22:04.425950+0000","last_fullsized":"2023-11-28T11:22:04.425950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:32.076790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.263216+0000","last_change":"2023-11-28T11:17:43.457143+0000","last_active":"2023-11-28T11:22:05.263216+0000","last_peered":"2023-11-28T11:22:05.263216+0000","last_clean":"2023-11-28T11:22:05.263216+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:22:05.263216+0000","last_undegraded":"2023-11-28T11:22:05.263216+0000","last_fullsized":"2023-11-28T11:22:05.263216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:01.594531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:05.257573+0000","last_change":"2023-11-28T11:17:43.457198+0000","last_active":"2023-11-28T11:22:05.257573+0000","last_peered":"2023-11-28T11:22:05.257573+0000","last_clean":"2023-11-28T11:22:05.257573+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:22:05.257573+0000","last_undegraded":"2023-11-28T11:22:05.257573+0000","last_fullsized":"2023-11-28T11:22:05.257573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:17:42.440830+0000","last_clean_scrub_stamp":"2023-11-28T11:17:42.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:13.538919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15128,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356712,"statfs":{"total":96636764160,"available":96621273088,"internally_reserved":0,"allocated":1392640,"data_stored":943512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.87,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.625}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.745},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.204,"5min":2.23,"15min":2.23},"last":0.81599999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.147,"15min":1.147},"last":0.70199999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.56499999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70699999999999996,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.745}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.625},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0269999999999999,"5min":1.627,"15min":1.627},"last":0.85299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.876,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.745,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13904,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357936,"statfs":{"total":96636764160,"available":96622526464,"internally_reserved":0,"allocated":794624,"data_stored":352180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94999999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85299999999999998,"5min":1.181,"15min":1.181},"last":0.95799999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.03},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":1.774,"15min":1.774},"last":1.1140000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96699999999999997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.048},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.137}]}]},{"osd":0,"up_from":14,"seq":60129542202,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14932,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356908,"statfs":{"total":96636764160,"available":96621473792,"internally_reserved":0,"allocated":1388544,"data_stored":942493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":1.052,"15min":1.052},"last":0.44},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47499999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.74399999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0249999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79100000000000004,"15min":0.80800000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2729999999999999,"5min":1.458,"15min":1.458},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.84199999999999997,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1639999999999999,"5min":1.591,"15min":1.591},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.64100000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.208,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.046,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.76200000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13716,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358124,"statfs":{"total":96636764160,"available":96622718976,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.58099999999999996,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63200000000000001,"5min":1.04,"15min":1.04},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58299999999999996,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":4.093,"15min":4.093},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68200000000000005,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":3.956,"15min":3.956},"last":0.627}]},{"osd":4,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76100000000000001,"15min":0.81899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0369999999999999,"5min":2.028,"15min":2.028},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.147,"15min":1.147},"last":1.1539999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.95199999999999996},"min":{"1min":0.58399999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.024,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":1.032},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0880000000000001,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":1.0489999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.90800000000000003,"15min":1.0249999999999999},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.177,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":1.1200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80500000000000005,"15min":0.85499999999999998},"min":{"1min":0.65900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":1.272,"15min":1.272},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.86899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.1910000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542201,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14036,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357804,"statfs":{"total":96636764160,"available":96622391296,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48799999999999999,"15min":0.54300000000000004},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59999999999999998,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":3.004,"15min":3.004},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57399999999999995,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69599999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":3.274,"15min":3.274},"last":0.443}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":1.0940000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.034,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.90800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.93300000000000005},"min":{"1min":0.66700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0860000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.032},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.077,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.073},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":1.006}]}]},{"osd":4,"up_from":14,"seq":60129542200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":14100,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357740,"statfs":{"total":96636764160,"available":96622325760,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97699999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.73499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96899999999999997,"5min":1.847,"15min":1.847},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":1.615,"15min":1.615},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.099,"5min":1.127,"15min":1.127},"last":0.63300000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.083,"5min":1.089,"15min":1.089},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.127,"5min":1.296,"15min":1.296},"last":0.502},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.212,"15min":1.212},"last":0.53000000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542201,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13908,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357932,"statfs":{"total":96636764160,"available":96622522368,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92900000000000005,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.72599999999999998},"min":{"1min":0.54800000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97699999999999998,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":2.661,"15min":2.661},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.79700000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.73299999999999998},"min":{"1min":0.53700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.754},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.83799999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86099999999999999,"5min":5.375,"15min":5.375},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.333,"15min":1.333},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:16.053 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.053 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:17:41.438817+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.054 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.055 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T11:17:42.440830+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=17, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=22, tm_sec=9, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:22:16.055 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:22:36.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:38.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.098+0000 7f05fd7ff700 1 Processor -- start 2023-11-28T11:22:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.098+0000 7f05fd7ff700 1 -- start start 2023-11-28T11:22:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05fd7ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05f80b0d80 0x7f05f815add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05fd7ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f815e1a0 con 0x7f05f8158200 2023-11-28T11:22:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05fd7ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f80cb180 con 0x7f05f8070c50 2023-11-28T11:22:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05fd7ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f815f070 con 0x7f05f80b0d80 2023-11-28T11:22:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05f6ffd700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05f8158200 0x7f05f80caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58396/0 (socket says 172.21.15.53:58396) 2023-11-28T11:22:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05f6ffd700 1 -- 172.21.15.53:0/4238098324 learned_addr learned my addr 172.21.15.53:0/4238098324 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05f77fe700 1 --2- 172.21.15.53:0/4238098324 >> v2:172.21.15.157:3300/0 conn(0x7f05f80b0d80 0x7f05f815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05f67fc700 1 -- 172.21.15.53:0/4238098324 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846925421 0 0) 0x7f05f815e1a0 con 0x7f05f8158200 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.100+0000 7f05f67fc700 1 -- 172.21.15.53:0/4238098324 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e40035b0 con 0x7f05f8158200 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f77fe700 1 -- 172.21.15.53:0/4238098324 >> v1:172.21.15.157:6789/0 conn(0x7f05f8070c50 legacy=0x7f05f80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f77fe700 1 -- 172.21.15.53:0/4238098324 >> v1:172.21.15.53:6789/0 conn(0x7f05f8158200 legacy=0x7f05f80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f77fe700 1 -- 172.21.15.53:0/4238098324 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80710f0 con 0x7f05f80b0d80 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f77fe700 1 --2- 172.21.15.53:0/4238098324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f80b0d80 0x7f05f815add0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5df936c9b18da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f67fc700 1 -- 172.21.15.53:0/4238098324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000e070 con 0x7f05f80b0d80 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f67fc700 1 -- 172.21.15.53:0/4238098324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000bb20 con 0x7f05f80b0d80 2023-11-28T11:22:38.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.101+0000 7f05f67fc700 1 -- 172.21.15.53:0/4238098324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0012430 con 0x7f05f80b0d80 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 -- 172.21.15.53:0/4238098324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f80b0d80 msgr2=0x7f05f815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/4238098324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f80b0d80 0x7f05f815add0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 -- 172.21.15.53:0/4238098324 shutdown_connections 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/4238098324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f80b0d80 0x7f05f815add0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 -- 172.21.15.53:0/4238098324 >> 172.21.15.53:0/4238098324 conn(0x7f05f80cb670 msgr2=0x7f05f80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 -- 172.21.15.53:0/4238098324 shutdown_connections 2023-11-28T11:22:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.102+0000 7f05fd7ff700 1 -- 172.21.15.53:0/4238098324 wait complete. 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.104+0000 7f05fd7ff700 1 Processor -- start 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.104+0000 7f05fd7ff700 1 -- start start 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f8070c50 0x7f05f80ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f8158200 0x7f05f80c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80aca20 con 0x7f05f80b0d80 2023-11-28T11:22:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80acc40 con 0x7f05f8158200 2023-11-28T11:22:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05fd7ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80acf20 con 0x7f05f8070c50 2023-11-28T11:22:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50442/0 (socket says 172.21.15.53:50442) 2023-11-28T11:22:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.105+0000 7f05effff700 1 -- 172.21.15.53:0/2127518550 learned_addr learned my addr 172.21.15.53:0/2127518550 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05f6ffd700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f8070c50 0x7f05f80ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05f77fe700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f8158200 0x7f05f80c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 -- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f8070c50 msgr2=0x7f05f80ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f8070c50 0x7f05f80ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 -- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f8158200 msgr2=0x7f05f80c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f8158200 0x7f05f80c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 -- 172.21.15.53:0/2127518550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0009a70 con 0x7f05f80b0d80 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05effff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f05e000c4e0 tx=0x7f05e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00052d0 con 0x7f05f80b0d80 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0005430 con 0x7f05f80b0d80 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00038c0 con 0x7f05f80b0d80 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f815e3f0 con 0x7f05f80b0d80 2023-11-28T11:22:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.106+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f81585d0 con 0x7f05f80b0d80 2023-11-28T11:22:38.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.107+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f815e6d0 con 0x7f05f80b0d80 2023-11-28T11:22:38.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.108+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05e0019030 con 0x7f05f80b0d80 2023-11-28T11:22:38.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.110+0000 7f05ef7fe700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 0x7f05d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.110+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f05e0084dd0 con 0x7f05f80b0d80 2023-11-28T11:22:38.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.110+0000 7f05f6ffd700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 0x7f05d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.111+0000 7f05f6ffd700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 0x7f05d806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05e80011e0 tx=0x7f05e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.114+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05e0051860 con 0x7f05f80b0d80 2023-11-28T11:22:38.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.311+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 --> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05f80ad6c0 con 0x7f05d80682f0 2023-11-28T11:22:38.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.314+0000 7f05ef7fe700 1 -- 172.21.15.53:0/2127518550 <== mgr.4105 v2:172.21.15.157:6832/132254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73036 (secure 0 0 0) 0x7f05f80ad6c0 con 0x7f05d80682f0 2023-11-28T11:22:38.317 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.316+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 msgr2=0x7f05d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.316+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 0x7f05d806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05e80011e0 tx=0x7f05e8009040 comp rx=0 tx=0).stop 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 msgr2=0x7f05f80ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f05e000c4e0 tx=0x7f05e000c8a0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 shutdown_connections 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f05d80682f0 0x7f05d806a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f8070c50 0x7f05f80ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f8158200 0x7f05f80c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 --2- 172.21.15.53:0/2127518550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05f80b0d80 0x7f05f80ad9e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 >> 172.21.15.53:0/2127518550 conn(0x7f05f80cb670 msgr2=0x7f05f8159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 shutdown_connections 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.317+0000 7f05fd7ff700 1 -- 172.21.15.53:0/2127518550 wait complete. 2023-11-28T11:22:38.320 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-28T11:22:38.337 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-28T11:22:37.005935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":113440,"kb_used_data":7392,"kb_used_omap":0,"kb_used_meta":105984,"kb_avail":754861280,"statfs":{"total":773094113280,"available":772977950720,"internally_reserved":0,"allocated":7569408,"data_stored":4003905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":108527616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002779"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:12.321596+0000","last_change":"2023-11-28T11:22:12.321573+0000","last_active":"2023-11-28T11:22:12.321596+0000","last_peered":"2023-11-28T11:22:12.321596+0000","last_clean":"2023-11-28T11:22:12.321596+0000","last_became_active":"2023-11-28T11:17:43.460043+0000","last_became_peered":"2023-11-28T11:17:43.460043+0000","last_unstale":"2023-11-28T11:22:12.321596+0000","last_undegraded":"2023-11-28T11:22:12.321596+0000","last_fullsized":"2023-11-28T11:22:12.321596+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:12.321499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:12.321499+0000","last_clean_scrub_stamp":"2023-11-28T11:22:12.321499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:13.717972+0000","scrub_duration":0.0036540359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:16.411191+0000","last_change":"2023-11-28T11:22:16.411142+0000","last_active":"2023-11-28T11:22:16.411191+0000","last_peered":"2023-11-28T11:22:16.411191+0000","last_clean":"2023-11-28T11:22:16.411191+0000","last_became_active":"2023-11-28T11:17:43.456833+0000","last_became_peered":"2023-11-28T11:17:43.456833+0000","last_unstale":"2023-11-28T11:22:16.411191+0000","last_undegraded":"2023-11-28T11:22:16.411191+0000","last_fullsized":"2023-11-28T11:22:16.411191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:16.411035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:16.411035+0000","last_clean_scrub_stamp":"2023-11-28T11:22:16.411035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:12.110935+0000","scrub_duration":0.0038801040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:12.202428+0000","last_change":"2023-11-28T11:22:12.202383+0000","last_active":"2023-11-28T11:22:12.202428+0000","last_peered":"2023-11-28T11:22:12.202428+0000","last_clean":"2023-11-28T11:22:12.202428+0000","last_became_active":"2023-11-28T11:17:43.543158+0000","last_became_peered":"2023-11-28T11:17:43.543158+0000","last_unstale":"2023-11-28T11:22:12.202428+0000","last_undegraded":"2023-11-28T11:22:12.202428+0000","last_fullsized":"2023-11-28T11:22:12.202428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:12.202277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:12.202277+0000","last_clean_scrub_stamp":"2023-11-28T11:22:12.202277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:22:32.324916+0000","scrub_duration":0.003544148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:15.370644+0000","last_change":"2023-11-28T11:22:15.370610+0000","last_active":"2023-11-28T11:22:15.370644+0000","last_peered":"2023-11-28T11:22:15.370644+0000","last_clean":"2023-11-28T11:22:15.370644+0000","last_became_active":"2023-11-28T11:17:43.457297+0000","last_became_peered":"2023-11-28T11:17:43.457297+0000","last_unstale":"2023-11-28T11:22:15.370644+0000","last_undegraded":"2023-11-28T11:22:15.370644+0000","last_fullsized":"2023-11-28T11:22:15.370644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:15.370393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:15.370393+0000","last_clean_scrub_stamp":"2023-11-28T11:22:15.370393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:38.783214+0000","scrub_duration":0.0037374930000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"66'119","reported_seq":219,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:18.742745+0000","last_change":"2023-11-28T11:22:18.742695+0000","last_active":"2023-11-28T11:22:18.742745+0000","last_peered":"2023-11-28T11:22:18.742745+0000","last_clean":"2023-11-28T11:22:18.742745+0000","last_became_active":"2023-11-28T11:17:42.454093+0000","last_became_peered":"2023-11-28T11:17:42.454093+0000","last_unstale":"2023-11-28T11:22:18.742745+0000","last_undegraded":"2023-11-28T11:22:18.742745+0000","last_fullsized":"2023-11-28T11:22:18.742745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'119","last_scrub_stamp":"2023-11-28T11:22:18.742602+0000","last_deep_scrub":"66'119","last_deep_scrub_stamp":"2023-11-28T11:22:18.742602+0000","last_clean_scrub_stamp":"2023-11-28T11:22:18.742602+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:12.391159+0000","scrub_duration":0.0079753389999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.701673+0000","last_change":"2023-11-28T11:22:26.701638+0000","last_active":"2023-11-28T11:22:26.701673+0000","last_peered":"2023-11-28T11:22:26.701673+0000","last_clean":"2023-11-28T11:22:26.701673+0000","last_became_active":"2023-11-28T11:17:43.456683+0000","last_became_peered":"2023-11-28T11:17:43.456683+0000","last_unstale":"2023-11-28T11:22:26.701673+0000","last_undegraded":"2023-11-28T11:22:26.701673+0000","last_fullsized":"2023-11-28T11:22:26.701673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:26.701527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:26.701527+0000","last_clean_scrub_stamp":"2023-11-28T11:22:26.701527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:15:14.001740+0000","scrub_duration":0.0040925830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:28.723558+0000","last_change":"2023-11-28T11:22:28.723495+0000","last_active":"2023-11-28T11:22:28.723558+0000","last_peered":"2023-11-28T11:22:28.723558+0000","last_clean":"2023-11-28T11:22:28.723558+0000","last_became_active":"2023-11-28T11:17:43.459891+0000","last_became_peered":"2023-11-28T11:17:43.459891+0000","last_unstale":"2023-11-28T11:22:28.723558+0000","last_undegraded":"2023-11-28T11:22:28.723558+0000","last_fullsized":"2023-11-28T11:22:28.723558+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:28.723318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:28.723318+0000","last_clean_scrub_stamp":"2023-11-28T11:22:28.723318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:07.253715+0000","scrub_duration":0.0044430470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:18.341047+0000","last_change":"2023-11-28T11:22:18.340993+0000","last_active":"2023-11-28T11:22:18.341047+0000","last_peered":"2023-11-28T11:22:18.341047+0000","last_clean":"2023-11-28T11:22:18.341047+0000","last_became_active":"2023-11-28T11:17:43.456347+0000","last_became_peered":"2023-11-28T11:17:43.456347+0000","last_unstale":"2023-11-28T11:22:18.341047+0000","last_undegraded":"2023-11-28T11:22:18.341047+0000","last_fullsized":"2023-11-28T11:22:18.341047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:18.340713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:18.340713+0000","last_clean_scrub_stamp":"2023-11-28T11:22:18.340713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:00.214524+0000","scrub_duration":0.0042302520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:22:18.884940+0000","last_change":"2023-11-28T11:22:18.884902+0000","last_active":"2023-11-28T11:22:18.884940+0000","last_peered":"2023-11-28T11:22:18.884940+0000","last_clean":"2023-11-28T11:22:18.884940+0000","last_became_active":"2023-11-28T11:17:43.456801+0000","last_became_peered":"2023-11-28T11:17:43.456801+0000","last_unstale":"2023-11-28T11:22:18.884940+0000","last_undegraded":"2023-11-28T11:22:18.884940+0000","last_fullsized":"2023-11-28T11:22:18.884940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:18.884788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:18.884788+0000","last_clean_scrub_stamp":"2023-11-28T11:22:18.884788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:25.603413+0000","scrub_duration":0.0037244700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15192,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356648,"statfs":{"total":96636764160,"available":96621207552,"internally_reserved":0,"allocated":1392640,"data_stored":943512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.91400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.87,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.056},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.99399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.0209999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69799999999999995,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.80600000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13652,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358188,"statfs":{"total":96636764160,"available":96622784512,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.204,"5min":2.23,"15min":2.23},"last":0.995}]},{"osd":1,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.147,"15min":1.147},"last":0.751}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.56499999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.871},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70699999999999996,"15min":0.67500000000000004},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82599999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.625},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0269999999999999,"5min":1.627,"15min":1.627},"last":0.95899999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.0569999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.876,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542207,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13904,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357936,"statfs":{"total":96636764160,"available":96622526464,"internally_reserved":0,"allocated":794624,"data_stored":352180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.373},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51900000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94999999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":1.024},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85299999999999998,"5min":1.181,"15min":1.181},"last":0.83099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":1.774,"15min":1.774},"last":0.86799999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.92300000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96699999999999997,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.99099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14932,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356908,"statfs":{"total":96636764160,"available":96621473792,"internally_reserved":0,"allocated":1388544,"data_stored":942493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":1.052,"15min":1.052},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.74399999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.75},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73699999999999999,"15min":0.70499999999999996},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0249999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79100000000000004,"15min":0.80800000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2729999999999999,"5min":1.458,"15min":1.458},"last":0.84899999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.84199999999999997,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1639999999999999,"5min":1.591,"15min":1.591},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.64100000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.208,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.046,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.0109999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13716,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358124,"statfs":{"total":96636764160,"available":96622718976,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.58099999999999996,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63200000000000001,"5min":1.04,"15min":1.04},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58299999999999996,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":4.093,"15min":4.093},"last":0.496},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68200000000000005,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":3.956,"15min":3.956},"last":0.66300000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76100000000000001,"15min":0.81899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0369999999999999,"5min":2.028,"15min":2.028},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.147,"15min":1.147},"last":0.749}]},{"osd":5,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.95199999999999996},"min":{"1min":0.58399999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.024,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0880000000000001,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.73299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.90800000000000003,"15min":1.0249999999999999},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.177,"5min":4.4580000000000002,"15min":4.4580000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80500000000000005,"15min":0.85499999999999998},"min":{"1min":0.65900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":1.272,"15min":1.272},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.86899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.70699999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542206,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14036,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357804,"statfs":{"total":96636764160,"available":96622391296,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48799999999999999,"15min":0.54300000000000004},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59999999999999998,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":3.004,"15min":3.004},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57399999999999995,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69599999999999995,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":3.274,"15min":3.274},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.034,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.873}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.93300000000000005},"min":{"1min":0.66700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0860000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.077,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.78900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":14100,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357740,"statfs":{"total":96636764160,"available":96622325760,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72599999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97699999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.86699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83099999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96899999999999997,"5min":1.847,"15min":1.847},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":1.615,"15min":1.615},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.099,"5min":1.127,"15min":1.127},"last":0.5}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.083,"5min":1.089,"15min":1.089},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.127,"5min":1.296,"15min":1.296},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.212,"15min":1.212},"last":0.623}]}]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13908,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357932,"statfs":{"total":96636764160,"available":96622522368,"internally_reserved":0,"allocated":798720,"data_stored":353144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92900000000000005,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.72599999999999998},"min":{"1min":0.54800000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.80900000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":3.3199999999999998,"15min":3.3199999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97699999999999998,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.77200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":2.661,"15min":2.661},"last":0.745},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.79300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.73299999999999998},"min":{"1min":0.53700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.82799999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86099999999999999,"5min":5.375,"15min":5.375},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.333,"15min":1.333},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":4.1859999999999999,"15min":4.1859999999999999},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:38.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:22:38.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.532+0000 7feb435ff700 1 Processor -- start 2023-11-28T11:22:38.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.532+0000 7feb435ff700 1 -- start start 2023-11-28T11:22:38.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.533+0000 7feb435ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 0x7feb3c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.533+0000 7feb435ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 0x7feb3c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.533+0000 7feb435ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0ce160 con 0x7feb3c161340 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.533+0000 7feb435ff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c0b5800 con 0x7feb3c0cd7c0 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.533+0000 7feb435ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0d1950 con 0x7feb3c15c9b0 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb40b9a700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb3c161340 0x7feb3c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58398/0 (socket says 172.21.15.53:58398) 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb40b9a700 1 -- 172.21.15.53:0/4003938876 learned_addr learned my addr 172.21.15.53:0/4003938876 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb4139b700 1 --2- 172.21.15.53:0/4003938876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 0x7feb3c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb41b9c700 1 --2- 172.21.15.53:0/4003938876 >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 0x7feb3c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb33fff700 1 -- 172.21.15.53:0/4003938876 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205499831 0 0) 0x7feb3c0ce160 con 0x7feb3c161340 2023-11-28T11:22:38.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb33fff700 1 -- 172.21.15.53:0/4003938876 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb280035b0 con 0x7feb3c161340 2023-11-28T11:22:38.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb4139b700 1 -- 172.21.15.53:0/4003938876 >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 msgr2=0x7feb3c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:38.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb4139b700 1 --2- 172.21.15.53:0/4003938876 >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 0x7feb3c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb4139b700 1 -- 172.21.15.53:0/4003938876 >> v1:172.21.15.53:6789/0 conn(0x7feb3c161340 legacy=0x7feb3c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb4139b700 1 -- 172.21.15.53:0/4003938876 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0d24f0 con 0x7feb3c15c9b0 2023-11-28T11:22:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.534+0000 7feb41b9c700 1 --2- 172.21.15.53:0/4003938876 >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 0x7feb3c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb4139b700 1 --2- 172.21.15.53:0/4003938876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 0x7feb3c0b4bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb38009840 tx=0x7feb38009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad40938c20332a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb33fff700 1 -- 172.21.15.53:0/4003938876 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38011830 con 0x7feb3c15c9b0 2023-11-28T11:22:38.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb33fff700 1 -- 172.21.15.53:0/4003938876 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38011990 con 0x7feb3c15c9b0 2023-11-28T11:22:38.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb33fff700 1 -- 172.21.15.53:0/4003938876 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38011d00 con 0x7feb3c15c9b0 2023-11-28T11:22:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb435ff700 1 -- 172.21.15.53:0/4003938876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 msgr2=0x7feb3c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.535+0000 7feb435ff700 1 --2- 172.21.15.53:0/4003938876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 0x7feb3c0b4bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb38009840 tx=0x7feb38009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 -- 172.21.15.53:0/4003938876 shutdown_connections 2023-11-28T11:22:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 --2- 172.21.15.53:0/4003938876 >> v2:172.21.15.157:3300/0 conn(0x7feb3c0cd7c0 0x7feb3c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 --2- 172.21.15.53:0/4003938876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c15c9b0 0x7feb3c0b4bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 -- 172.21.15.53:0/4003938876 >> 172.21.15.53:0/4003938876 conn(0x7feb3c072600 msgr2=0x7feb3c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 -- 172.21.15.53:0/4003938876 shutdown_connections 2023-11-28T11:22:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.536+0000 7feb435ff700 1 -- 172.21.15.53:0/4003938876 wait complete. 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.537+0000 7feb435ff700 1 Processor -- start 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.537+0000 7feb435ff700 1 -- start start 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c0cd7c0 0x7feb3c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3c161340 0x7feb3c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c14db00 con 0x7feb3c15c9b0 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1f5ba0 con 0x7feb3c161340 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.538+0000 7feb435ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1f5e80 con 0x7feb3c0cd7c0 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50454/0 (socket says 172.21.15.53:50454) 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 -- 172.21.15.53:0/2539148657 learned_addr learned my addr 172.21.15.53:0/2539148657 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb4139b700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c0cd7c0 0x7feb3c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb41b9c700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3c161340 0x7feb3c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 -- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c0cd7c0 msgr2=0x7feb3c0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c0cd7c0 0x7feb3c0b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 -- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3c161340 msgr2=0x7feb3c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3c161340 0x7feb3c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 -- 172.21.15.53:0/2539148657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34009a70 con 0x7feb3c15c9b0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.539+0000 7feb40b9a700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c420 tx=0x7feb2c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.540+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c005d70 con 0x7feb3c15c9b0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.540+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c00ce20 con 0x7feb3c15c9b0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.540+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0072d0 con 0x7feb3c15c9b0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.540+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb380094a0 con 0x7feb3c15c9b0 2023-11-28T11:22:38.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.540+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c0d1300 con 0x7feb3c15c9b0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.541+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c0b5520 con 0x7feb3c15c9b0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.542+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb2c019030 con 0x7feb3c15c9b0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.543+0000 7feb31ffb700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 0x7feb2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.543+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7feb2c084b20 con 0x7feb3c15c9b0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.543+0000 7feb4139b700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 0x7feb2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.544+0000 7feb4139b700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 0x7feb2006a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb38009fd0 tx=0x7feb3801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.547+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb2c04b5e0 con 0x7feb3c15c9b0 2023-11-28T11:22:38.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.751+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7feb3c0d15e0 con 0x7feb3c15c9b0 2023-11-28T11:22:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.754+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feb2c051660 con 0x7feb3c15c9b0 2023-11-28T11:22:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.755+0000 7feb31ffb700 1 -- 172.21.15.53:0/2539148657 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7feb2c017020 con 0x7feb3c15c9b0 2023-11-28T11:22:38.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 msgr2=0x7feb2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 0x7feb2006a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb38009fd0 tx=0x7feb3801c040 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 msgr2=0x7feb3c14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c420 tx=0x7feb2c00c730 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 shutdown_connections 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7feb200682f0 0x7feb2006a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb3c0cd7c0 0x7feb3c0b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb3c161340 0x7feb3c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 --2- 172.21.15.53:0/2539148657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c15c9b0 0x7feb3c14d080 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 >> 172.21.15.53:0/2539148657 conn(0x7feb3c072600 msgr2=0x7feb3c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 shutdown_connections 2023-11-28T11:22:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.758+0000 7feb435ff700 1 -- 172.21.15.53:0/2539148657 wait complete. 2023-11-28T11:22:38.778 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:22:38.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.979+0000 7f7ede3ff700 1 Processor -- start 2023-11-28T11:22:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.979+0000 7f7ede3ff700 1 -- start start 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ede3ff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ede3ff700 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed80d1950 con 0x7f7ed8161340 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ede3ff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed80d24f0 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ede3ff700 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed80b5800 con 0x7f7ed815c9b0 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ed7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ed77fe700 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7ed8161340 0x7f7ed80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58410/0 (socket says 172.21.15.53:58410) 2023-11-28T11:22:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ed77fe700 1 -- 172.21.15.53:0/2504627182 learned_addr learned my addr 172.21.15.53:0/2504627182 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-11-28T11:22:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.981+0000 7f7ed7fff700 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50460/0 (socket says 172.21.15.53:50460) 2023-11-28T11:22:38.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed7fff700 1 -- 172.21.15.53:0/2504627182 >> v1:172.21.15.157:6789/0 conn(0x7f7ed80cd7c0 legacy=0x7f7ed80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed7fff700 1 -- 172.21.15.53:0/2504627182 >> v1:172.21.15.53:6789/0 conn(0x7f7ed8161340 legacy=0x7f7ed80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed7fff700 1 -- 172.21.15.53:0/2504627182 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80b5520 con 0x7f7ed815c9b0 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed6ffd700 1 -- 172.21.15.53:0/2504627182 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381651326 0 0) 0x7f7ed80d1950 con 0x7f7ed8161340 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed7fff700 1 --2- 172.21.15.53:0/2504627182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009840 tx=0x7f7ec8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49766b81553f683d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed6ffd700 1 -- 172.21.15.53:0/2504627182 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8011830 con 0x7f7ed815c9b0 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed6ffd700 1 -- 172.21.15.53:0/2504627182 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7ec8011990 con 0x7f7ed815c9b0 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.982+0000 7f7ed6ffd700 1 -- 172.21.15.53:0/2504627182 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8011d40 con 0x7f7ed815c9b0 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 -- 172.21.15.53:0/2504627182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed815c9b0 msgr2=0x7f7ed80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/2504627182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009840 tx=0x7f7ec8009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 -- 172.21.15.53:0/2504627182 shutdown_connections 2023-11-28T11:22:38.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/2504627182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed815c9b0 0x7f7ed80b4bf0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 -- 172.21.15.53:0/2504627182 >> 172.21.15.53:0/2504627182 conn(0x7f7ed8072600 msgr2=0x7f7ed80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:38.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 -- 172.21.15.53:0/2504627182 shutdown_connections 2023-11-28T11:22:38.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.983+0000 7f7ede3ff700 1 -- 172.21.15.53:0/2504627182 wait complete. 2023-11-28T11:22:38.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.985+0000 7f7ede3ff700 1 Processor -- start 2023-11-28T11:22:38.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.985+0000 7f7ede3ff700 1 -- start start 2023-11-28T11:22:38.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed815c9b0 0x7f7ed81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed8161340 0x7f7ed81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80ca470 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8157d30 con 0x7f7ed8161340 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ede3ff700 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80c9b70 con 0x7f7ed815c9b0 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50468/0 (socket says 172.21.15.53:50468) 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ed7fff700 1 -- 172.21.15.53:0/644065121 learned_addr learned my addr 172.21.15.53:0/644065121 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7ed77fe700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed815c9b0 0x7f7ed81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.986+0000 7f7edc99c700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed8161340 0x7f7ed81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 -- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed815c9b0 msgr2=0x7f7ed81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed815c9b0 0x7f7ed81572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 -- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed8161340 msgr2=0x7f7ed81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed8161340 0x7f7ed81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 -- 172.21.15.53:0/644065121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80094a0 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed7fff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80090b0 tx=0x7f7ec8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80045d0 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7ec801f070 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed80d15e0 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.987+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80038a0 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.988+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed80d1300 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.989+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed80b5130 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.990+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7ec800f050 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.990+0000 7f7ed4ff9700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 0x7f7ec406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.990+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7ec8021070 con 0x7f7ed80cd7c0 2023-11-28T11:22:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.991+0000 7f7ed77fe700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 0x7f7ec406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:38.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.991+0000 7f7ed77fe700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 0x7f7ec406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007b10 tx=0x7f7ecc007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:38.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:38.995+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7ec8053eb0 con 0x7f7ed80cd7c0 2023-11-28T11:22:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.242+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ed80ca5b0 con 0x7f7ed80cd7c0 2023-11-28T11:22:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.243+0000 7f7ed4ff9700 1 -- 172.21.15.53:0/644065121 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3533 (secure 0 0 0) 0x7f7ec8015de0 con 0x7f7ed80cd7c0 2023-11-28T11:22:39.245 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-28T11:22:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/1688369224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1688369224},{"type":"v1","addr":"172.21.15.157:6836","nonce":1688369224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2599501188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2599501188},{"type":"v1","addr":"172.21.15.53:6835","nonce":2599501188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/551905428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":551905428},{"type":"v1","addr":"172.21.15.53:6834","nonce":551905428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":55325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3591597468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3591597468},{"type":"v1","addr":"172.21.15.157:6837","nonce":3591597468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-11-28T11:22:39.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 msgr2=0x7f7ec406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 0x7f7ec406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007b10 tx=0x7f7ecc007a60 comp rx=0 tx=0).stop 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 msgr2=0x7f7ed81e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80090b0 tx=0x7f7ec8004750 comp rx=0 tx=0).stop 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 shutdown_connections 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:6832/132254,v1:172.21.15.157:6833/132254] conn(0x7f7ec40682f0 0x7f7ec406a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed8161340 0x7f7ed81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed80cd7c0 0x7f7ed81e7150 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.245+0000 7f7ede3ff700 1 --2- 172.21.15.53:0/644065121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed815c9b0 0x7f7ed81572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.246+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 >> 172.21.15.53:0/644065121 conn(0x7f7ed8072600 msgr2=0x7f7ed815a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.246+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 shutdown_connections 2023-11-28T11:22:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-28T11:22:39.246+0000 7f7ede3ff700 1 -- 172.21.15.53:0/644065121 wait complete. 2023-11-28T11:22:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 35 2023-11-28T11:22:39.266 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:22:39.266 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:22:39.267 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:22:39.267 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.281 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:22:39.281 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:22:39.281 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.335 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:22:39.336 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:22:39.336 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.352 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:22:39.353 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:22:39.353 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.409 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:22:39.410 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:22:39.410 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:22:39.410 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.455 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:22:39.455 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:22:39.455 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.497 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:22:39.498 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:22:39.498 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.541 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:22:39.541 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:22:39.542 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.594 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:22:39.594 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:22:39.594 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.639 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:22:39.639 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:22:39.640 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.684 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:22:39.685 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:22:39.685 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.730 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:22:39.731 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:22:39.731 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.778 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:22:39.778 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:22:39.778 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:22:39.778 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.839 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:22:39.839 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:22:39.839 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.904 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:22:39.905 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:22:39.905 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:22:39.905 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.925 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:22:39.926 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:22:39.926 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:39.944 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:22:39.945 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:22:39.945 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:40.004 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:22:40.004 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:22:40.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:22:40.090 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:22:40.090 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:22:40.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:22:40.374 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:22:40.496 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:22:40.497 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:22:40.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:22:40.612 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:22:40.746 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:22:40.746 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:22:40.979 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:22:40.980 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:22:41.097 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:22:41.097 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:22:41.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:22:41.216 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:22:41.357 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:22:41.358 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/data/mon.a.tgz 2023-11-28T11:22:41.358 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2023-11-28T11:22:41.404 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.fcoTIm4Kan 2023-11-28T11:22:41.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.fcoTIm4Kan 2023-11-28T11:22:41.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.fcoTIm4Kan 2023-11-28T11:22:41.721 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.fcoTIm4Kan is 395KB 2023-11-28T11:22:41.826 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.fcoTIm4Kan 2023-11-28T11:22:41.869 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/data/mon.b.tgz 2023-11-28T11:22:41.870 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-11-28T11:22:41.915 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.ofj5M2iAJ8 2023-11-28T11:22:41.916 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ofj5M2iAJ8 2023-11-28T11:22:42.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.ofj5M2iAJ8 2023-11-28T11:22:42.260 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.ofj5M2iAJ8 is 511KB 2023-11-28T11:22:42.361 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.ofj5M2iAJ8 2023-11-28T11:22:42.404 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/data/mon.c.tgz 2023-11-28T11:22:42.404 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-11-28T11:22:42.487 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.GZSS3YK8Hr 2023-11-28T11:22:42.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.GZSS3YK8Hr 2023-11-28T11:22:42.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.GZSS3YK8Hr 2023-11-28T11:22:42.817 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.GZSS3YK8Hr is 510KB 2023-11-28T11:22:42.915 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.GZSS3YK8Hr 2023-11-28T11:22:42.958 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:22:42.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:22:42.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:22:43.067 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:22:43.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:22:43.072 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:22:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:22:43.148 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:22:43.165 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:22:43.167 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi053/crash 2023-11-28T11:22:43.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:22:43.236 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi157/crash 2023-11-28T11:22:43.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:22:43.304 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:22:43.305 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:43.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:43.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:43.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:43.548 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:22:43.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:22:43.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:22:45.473 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:22:45.474 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi053/log 2023-11-28T11:22:45.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:22:45.985 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi157/log 2023-11-28T11:22:45.986 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:22:46.229 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:22:46.250 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:22:46.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:22:46.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:22:46.377 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:22:46.388 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:22:46.388 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:22:46.393 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:22:46.429 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-28T11:22:46.434 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-11-28T11:22:46.437 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:22:46.437 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-28T11:22:46.437 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +773us[ +775us] +/- 89ms 2023-11-28T11:22:46.438 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 -627us[ -625us] +/- 63ms 2023-11-28T11:22:46.438 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +155us[ +155us] +/- 82ms 2023-11-28T11:22:46.438 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:22:46.441 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:22:46.441 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-11-28T11:22:46.442 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +781us[ +781us] +/- 89ms 2023-11-28T11:22:46.442 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -617us[ -600us] +/- 63ms 2023-11-28T11:22:46.442 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +182us[ +199us] +/- 81ms 2023-11-28T11:22:46.442 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:22:46.443 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:22:46.452 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:22:46.453 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:22:46.462 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:22:46.657 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:22:46.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:22:46.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:46.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:22:46.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:46.976 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:22:46.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:22:46.996 INFO:teuthology.task.internal:Duration was 1312.031566 seconds 2023-11-28T11:22:46.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:22:47.005 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:22:47.006 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:22:47.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:22:47.125 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:22:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:22:47.357 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:22:47.357 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2023-11-28T11:22:47.357 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:22:47.417 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-11-28T11:22:47.417 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:22:47.477 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:22:47.478 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:22:47.482 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:22:47.568 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:22:47.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:22:47.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:22:47.704 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:22:47.714 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:22:47.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:22:47.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:22:47.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:22:47.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:22:47.868 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:22:47.932 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2023-11-28T11:22:47.936 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-11-28T11:22:47.972 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:22:48.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:48.014 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:22:48.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:48.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:22:48.066 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:22:48.067 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi053 2023-11-28T11:22:48.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:22:48.146 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958/remote/smithi157 2023-11-28T11:22:48.147 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:22:48.226 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:22:48.226 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:22:48.230 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:22:48.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:22:48.287 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:22:48.287 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:22:48.297 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:22:48.297 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:22:48.314 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:22:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:22 /home/ubuntu/cephtest 2023-11-28T11:22:48.358 INFO:teuthology.orchestra.run.smithi157.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:22 /home/ubuntu/cephtest 2023-11-28T11:22:48.361 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:22:48.372 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:22:48.392 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:22:48.417 DEBUG:teuthology.nuke:shortname: smithi053 2023-11-28T11:22:48.418 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:22:48.442 DEBUG:teuthology.nuke:shortname: smithi157 2023-11-28T11:22:48.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:22:48.446 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:57.408838', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:22:48.470 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:57.410360', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:22:48.494 INFO:teuthology.orchestra.console.smithi053:Power off 2023-11-28T11:22:48.495 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:22:48.518 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-11-28T11:22:48.519 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:48.534 INFO:teuthology.orchestra.console.smithi157:Power off 2023-11-28T11:22:48.534 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:22:48.544 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-28T11:22:48.555 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-11-28T11:22:48.555 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:48.577 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-11-28T11:22:52.545 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:52.578 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:52.655 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-28T11:22:52.688 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-11-28T11:22:56.658 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:56.689 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:22:56.769 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-11-28T11:22:56.799 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-11-28T11:22:56.871 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-11-28T11:22:56.900 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-11-28T11:22:57.006 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1312.0315656661987 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e54fae1fa2b74ba7a1845d8c80bfcb76 status: fail success: false 2023-11-28T11:22:57.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:22:57.084 INFO:teuthology.run:FAIL